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technology. 
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Exploring  Qualitative  Methodologies  in  Online  Learning  Environments 


Mary  Beth  Bianco 
Alison  A.  Carr-Chellman 
Penn  State  University 


Abstract 

Qualitative  inquiry  is  rich  in  personal  interaction  between  participant  and  researcher.  The  researcher  is 
an  instrument  (Creswell,  1998)  and  the  participant  an  active  sharer  in  the  process  and  in  some  cases  the  final 
interpretation  of  the  inferences,  as  well  The  qualitative  methodological  processes,  particularly  those  considered  the 
three  legs  of  triangulation  — interview,  observation  and  document  analysis  - vary  in  the  degree  and  type  of 
communicative  aspects  in  specific  contexts.  The  various  methods  used  in  collecting  and  interpreting  qualitative  data 
are  best  used  together  - multiple  methods.  (Denzin  and  Lincoln,  1998;  Creswell,  1998)  Using  more  than  one  gives 
the  researcher  an  avenue  to  corroborate  findings  and  add  a deeper  dimension  to  the  study.  For  example,  the 
probing  of  the  interview  combined  with  the  interpretation  of  reports  and  the  interaction  of  an  observation  form  a 
more  complete  picture  from  which  themes  and  issues  emerge,  allowing  the  researcher  to  tell  the  story  of  the 
participants  more  accurately  and  completely.  (Denzin  and  Lincoln,  1998)  Will  the  story  be  affected  with  the  advent 
of  new  technological  methods  of  not  only  collecting  the  data,  but  the  data  itself? 

This  discussion  seeks  to  consider  the  issues  related  to  conducting  qualitative  inquiries  in  online  settings 
and  the  implications  for  using  electronic  means  in  interview  and  observation.  Consideration  needs  to  be  given  to 
the  ramifications  of  online  interview  techniques.  “Face-to-face"  may  not  be  “face-to-face"  in  online  environments 
but  may  be  only  available  other  synchronous  forms,  such  as,  telephone  or  online  discussion.  The  implications  for 
electronic  discussion  formats  for  conducting  interviews  will  also  be  discussed. 

Triangulation,  in  the  most  accepted  definition,  then,  may  be  implausible  due  to  the  fact  that  the  interview  or 
observational  techniques  are  not  logistically  feasible  or  probable.  Observation  in  natural  settings  is  foundational 
to  the  integrity  of  the  observation  but  should  be  investigated  for  workability  as  well  as  rectitude.  (Creswell,  1998). 
Exploration  into  the  implications  for  observation  by  electronic  means  should  be  conducted  to  determine  the 
trustworthiness  of  this  new  form  of  data  collection. 

The  methodology  issues  discussed  here  are  those  that  relate  to  the  interview  and  observational  legs  of 
triangulation  in  conjunction  with  possible  avenues  for  document  analysis.  Various  strategies  for  testing  online 
qualitative  inquires  will  be  considered  and  discussed  related  to  their  potential  value  to  the  inquiry  process.  The 
inquiry  in  this  case  centers  on  methodological  implications  for  data  collection  and  their  impact  on  the  qualitative 
process  in  online  situations  or  by  electronic  means. 

Introduction 

There  are  a growing  number  of  research  studies  being  conducted  on  distance  education,  in  particular,  online 
learning  environments  (Schlough  & Bhuripanyo,  1998;  Hengni,  1998;Thompson  and  Nay,  1999;  French,  1999; 
Moon,  1998;  Mclssac,  et.  al,  1999;  Truman-Davis  and  Hartman,  1998;  Gunawardena,  et.  ah,  1998;  Levin  and  Ben- 
Jacob,  1998;  Hiltz,  1998;  Chen  & Mashhadi,  1998;  Jannasch-Pennell,  et.  ah,  1999;  McFerrin,  1999;  Donaldson  and 
Tomson,  1999;  Yong,  1998).  Online  or  web-based  course  delivery  is  becoming  more  and  more  prevalent  as  an 
integral  part  of  today’s  college  curriculum.  Instructors  and  university  professors  are  providing  more  web  support 
and  universities  are  soliciting  students  to  participate  in  online  coursework  in  the  “anytime,  anywhere”  model  of 
instructional  delivery.  (Gladieux  and  Swail,  1999,  Dunn,  2000). 

As  a result  of  the  increased  interest  in  research  in  the  area  of  online  learning,  there  has  been  a subsequent 
increase  in  the  interest  in  the  use  of  qualitative  methods  to  gain  a deeper  understanding  of  online  learning 
environments.  In  particular,  there  have  been  a large  number  of  studies  which  examined,  qualitatively,  the  student 
perceptions  of  online  learning  experiences  (e.g.,  McNeil  & Robin,  2000;  Mclsaac,  et.ah,  2000).  However,  there  has 
not  yet  been  adequate  exploration  of  the  use  of  qualitative  methods  in  online  learning  environment  inquiry. 

In  an  effort  to  understand  student's  perceptions  of  their  learning  in  this  emerging  environment,  much 
research  is  being  conducted  in  connection  to  online  learning.  Among  the  current  topics  of  interest  are  studies  which 
compare  online  to  traditional  delivery  of  the  same  course  material;  effectiveness  of  learning;  student  perceptions; 
fostering  collaboration;  and  various  course  delivery  models,.(e.g.,  Diaz  and  Cartnal,  2000;  Ryan,  2000). 
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Online  learning  has  both  its  advocates  and  its  detractors.  These  new  technologies  in  instructional  delivery 
carry  both  support  for  and  definite  opinions  against  online  learning  environments  and  their  potential  effect(s)  on 
learning.  If  learning  is  at  the  heart  of  our  choices  regarding  instructional  delivery  options,  researchers  and  educators 
are  obligated  to  fully  understand  student  experiences  in  many  types  of  learning  situations.  Student  perceptions  and 
experiences  should  be  priorities  in  the  design  process  and  should  be  strongly  considered  prior  to  media  selection 
decisions.  It  is  difficult  not  to  design  for  the  "delivery  method  d'jour",  however  no  matter  how  glizty  or  well-funded 
online  education  options  may  be,  their  impact  on  learners'  experiences  as  understood  through  qualitative  inquiry 
should  be  carefully  considered  by  all  practicing  instructional  designers. 

In  an  attempt  to  understand  the  culture  of  online-learning,  qualitative  methodology  is  a natural  choice  with 
regard  to  research  design.  The  tenets  of  phenomenology  (Denzin  and  Lincoln,  1998)  lead  us  to  deeper 
understandings  of  the  experiences  of  the  participants  and  the  interpretation  that  informs  our  understandings. 
Phenomenological  research  assumptions  carry  with  them  the  importance  of  the  voice  of  the  participant,  their 
descriptions  of  their  experiences,  their  reflections  on  their  lived  experiences.  (Creswell,  1998;  Moustakas,  1994).  In 
considering  online  learning  environments,  these  experiences  are  critical  in  an  attempt  to  gain  deeper  understandings. 
This  paper  is  an  attempt  to  discuss  some  of  the  potential  implications  of  doing  qualitative  research  with  online 
populations. 

Distance  Education  - Attitudes  and  Perceptions 

David  Noble’s  (2000)  outcry  against  online  learning  and  technology  is  largely  aimed  at  the  politics  of 
technology.  He  argues  that  the  technology  itself  is  indeed  value-laden  and  the  demand  for  online  education  is  non- 
existent, created  by  university  administrators.  (Young,  2000)  He  suggests  that  the  motivations  for  online  learning 
are  nothing  more  than  greed  on  the  part  of  university  administrators  and  should  be  resisted  by  faculty  in  the  same 
ways  that  Luddites  resisted  the  new  technologies  that  they  believed  would  put  their  own  children  out  of  future  work. 
Others  (Bowers,  1999;  Woody,  1999)  also  argue  that  while  the  Internet  is  an  excellent  dispenser  of  information, 
education  is  not  merely  the  acquisition  of  information.  By  making  learning  primarily  about  the  acquisition  of 
information,  there  is  the  potential  to  commodify  higher  education— learning  becomes  a product  for  sale  rather  than 
the  experience  of  learning  and  growth.  Like  Noble,  many  are  concerned  for  the  financial  emphasis  placed  on 
online  learning  environments  with  a lack  of  focus  on  “learning”.  (Noble,  1998;  Woody, 1998). 

Many  supporters  of  online  education  (Dunn,  2000;  Dede;  1997).  envision  a future  where  students  will  come 
to  universities  only  for  the  social  aspects  of  higher  education,  such  as  sports  or  other  non-academic  activities.  In  a 
report  prepared  by  the  College  Board,  predictions  about  the  future  of  online  learning  environments  included  the 
demise  of  the  university,  as  we  know  it,  within  30  years.  (Gladieux  and  Swail,  1999;Dunn,  2000).  Gladieux  and 
Swail  go  on  to  cite  examples  of  complete  degree  programs  offered  by  “traditional”  universities  online  and  refers  to 
them  as  leaders  in  the  electronic  market.  While  considerations  need  to  be  made  about  the  access  to  these  new 
technologies  and  the  target  populations  who  benefit  from  technological  advancement,  other  issues  should  be 
considered  as  well.  The  need  exists  for  a qualitative  examination  of  online  instruction.  Collaboration,  social 
interaction  and  relationship  building  activities  that  exist  in  face-to-face  instruction  have  been  addressed  somewhat  in 
design  literature 

Qualitative  methodologies  are  social  at  the  core  (Creswell,  1998;Denzin  and  Lincoln,  1998).  There  has 
always  been  an  understanding  that  part  of  the  qualitative  inquiry  experience  is  "being  in  the  field."  (Wolcott,  1995). 
Doing  fieldwork,  in  which  the  inquirer  worked  in  a natural  setting  to  collect  stories  by  observing,  examining 
documents,  and  interviewing  indigenous  populations,  has  always  been  a cornerstone  of  the  qualitative  research 
experience.(Lincoln  and  Guba,  1985).  What  is  the  implication  of  doing  qualitative  research  in  online  settings? 
Where  is  the  field?  Where  is  it  located?  Is  it  electronic?  Collecting  data  at  a distance  begs  the  question  - can 
qualitative  research  occur  in  the  exploration  of  online  learning  and  if  so,  how  will  that  experience  be  defined?  Will 
qualitative  research  lose  a certain  social  quality  when  we  translate  these  methods  from  naturalistic  inquiry  (Lincoln 
& Guba)  into  e-inquiry?  Will  the  researchers  enjoy  the  experience  of  doing  qualitative  research  in  an  e-field  as 
much  as  they  enjoy  being  in  a physical  field?  Will  there  be  some  important  essence  of  the  stories  that  are  lost  by 
doing  qualitative  research  electronically?  Is  there  any  way  to  do  qualitative  research  among  online  populations  in  a 
face-to-face  mode  and  what  is  gained  or  lost  in  this  choice?  And  what  of  the  qualitative  methods  themselves?  Will 
the  traditionally  separate  methods  of  interview,  observation  and  document  analysis  become  blurred  in  online 
settings?  If  an  interview  is  conducted  in  a chat  mode,  is  it  a document  to  be  analyzed  or  an  interview?  Clearly  we 
cannot  answer  all  of  these  questions  in  any  complete  way  here,  we  raise  them  as  serious  considerations  for  those 
engaged  in  qualitative  research  among  online  populations  as  well  as  for  those  who  supervise  students  who  wish  to 
pursue  qualitative  inquiry  projects  in  online  education. 
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Qualitative  Research  Epistemological  Viewpoints 

Qualitative  research  is  conducted  in  an  effort  to  understand  the  experiences  and  attitudes  of  people  in 
contextually  bound  settings.  Purposes  of  qualitative  research  include  inquiry  in  natural  rather  than  contrived  settings 
and  the  understanding  the  ontological  viewpoints  of  the  participants.  Perspective,  perception  and  experience  are 
epistemological  notions  that  are  sought  in  the  interpretation  and  understanding  of  specific  contexts  or  situations. 
Because  we  wish  to  understand  perceptions,  experiences,  and  perspectives  of  learners  in  online  environments,  there 
is  a viable  match  between  qualitative  inquiry  and  online  learning  environments.  However,  online  qualitative  inquiry 
methodology  has  not  been  clearly  delineated  or  explicated  nor  is  it  definitively  prescribed  in  the  qualitative  inquiry 
literature.  In  some  ways  we  have  gotten  ahead  of  ourselves.  We  are  using  qualitative  inquiry  because  it  appears  to 
be  an  epistemological  match  with  our  goals  for  understanding  online  learning  environments  prior  to  truly  exploring 
the  impacts  of  this  application  on  the  field  of  qualitative  inquiry  and  the  underlying  epistemological  foundations  on 
which  that  discipline  is  built. 

The  discussion  that  follows  is  an  overview  of  the  three  most  common  qualitative  data  collection  methods 
(interviewing,  observations,  and  document  analysis)  and  their  relevance  to  the  understanding  of  online  learning 
environments.  We  explore  some  of  the  implications  of  using  qualitative  methods  and  base  these  implications  on  our 
own  experiences  conducting  qualitative  studies  in  online  environments. 

Qualitative  Methodology 

Interviewing 

Interviews  and  observations  are  used  as  key  methods  of  recording  people's  experiences,  perceptions  and 
attitudes  in  qualitative  inquiry.  (Creswell,  1998).  Interviews  range  in  type  and  length  and  are  used  for  different 
purposes  but  are  present  in  virtually  all  qualitative  traditions.  Interviewees  are  often  selected  utilizing  purposeful 
sampling  processes  and  are  contacted  in  many  different  ways.  Telephone  interviews  are  conducted  when  access  to 
participants  in  a face-to-face  environment  are  hindered  in  some  way.  (Bruce,  1979;  de  Leeuw,  1992).  Telephone 
interviews  carry  the  advantage  that  the  participant  can  be  interviewed  at  a remote  location  saving  travel  time  and 
money,  but  the  researcher  loses  the  opportunity  to  observe  nonverbal  communicative  actions.  Focus  group 
interviews  are  used  in  situations  where  the  interaction  among  participants  is  determined  to  be  beneficial  for  the 
inquiry.  Ideal  interview  situations  are  personal,  face-to-face  contexts  where  the  interviewer  and  the  interviewee 
have  the  opportunity  to  interact  in  an  open  atmosphere  and  establish  rapport  (Fontana  & Frey,  1994).  Typically, 
qualitative  interviews  are  recorded  either  by  audio  or  video  and  are  transcribed  for  the  analysis  of  data.  There  are  a 
number  of  ways  of  approaching  interviews  from  open-ended  to  highly  structured  interview  protocols. 

Computer-mediated  interviewing  (CMI)  techniques  add  quite  another  dimension.  Media  utilized  to 
conducte  CMI's  include  synchronous  chat,  email  and  discussion  forums.  As  with  telephone  interviews,  in  CMI, 
face-to-face  contact  is  lost.  Voice  tone  and  inflection,  hesitation  or  eagerness,  and  other  audible  indications  that  lead 
to  deeper  understandings  that  are  not  possible  through  CMI  are  lost.  While  not  without  benefit,  interviews 
conducted  via  chat  may  arouse  questions  regarding  the  conveyance  of  meaning  since  voice  and  body  language  are 
absent.  On  the  other  hand,  little  is  left  unquestioned  in  terms  of  specific  data  because  it  is  necessary  to  ask  for 
continual  clarification  from  the  participant  throughout  the  CMI. 


Face-to-Face 

Telephone 

Computer-mediated 

Visibility  of  facial 
expressions 

Absence  of  facial 

expressions  and  body 
language 

Absence  of  facial 

expressions  and  body 
language 

Personal  qualities  of 
establishing  rapport  with 
the  participant 

Voice  tone  assistance  in 
establishing  rapport 

Absence  of  personal 
contact  to  aid  in 
establishing  relationship 
with  participant(s). 

Communication  is  more 
natural  to  most  people  in 
conversation  form 

Conversational  tones  are 
possible 

Travel  time  required 

Travel  time  saved 

Travel  time  saved 

Transciption  costs 

incurred 

T ranscription  costs 

incurred 

Transcription  costs 

saved 

Travel  costs  incurred 

Telephone  costs  may  be 

Typically  no  or  low  cost 
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incurred  depending  on 
type  of  long  distance 
utilized 

via  internet  hookup 

Opportunities  for  on-site 
experiences  are 

preserved 

Opportunities  for  on-site 
experiences  lost 

Opportunities  for  on-site 
experiences  lost 

Ability  to  interview  a 
limited  number  of 

respondents 

Ability  to  interview 
more  respondents 

Ability  to  interview 
many  respondents 

including  those  abroad, 
disabled,  elderly, 

housebound 

Table  1:  Comparison  of  interview  data  collection  modalities 

It  is  possible  to  see  from  Table  1 a comparison  of  the  advantages  and  disadvantages  of  the  three  possible  media  for 
conducting  qualitative  interviews.  Obviously  the  primary  advantage  of  using  the  telephone  or  the  CMI  options  are 
resource  related,  e.g.,  time  and  money  to  travel  to  remote  sites.  However,  there  is  no  clear  indication  of  precisely 
how  much  is  lost  in  terms  of  the  deep  understanding  of  the  stories  being  told  by  the  respondents  by  losing  their 
affective,  interpersonal,  voice  and  body  reactions  to  questions.  It  is  not  clear  how  much  data  may  be  lost  by  the  very 
different  type  of  rapport  that  is  established  online  or  by  telephone  and  it  is  almost  impossible  to  quantify  or  qualify 
this  loss  because  qualitative  data  is  so  individualized  that  there  is  no  easy  comparison  study  that  can  be  conducted  to 
establish  what  is  truly  lost  and/or  gained  in  these  three  environments.  We  believe  that  these  decisions  are  likely  to 
be  made  too  often  because  of  cost  considerations.  Rather  than  seeking  additional  resources  to  allow  the  researcher 
to  be  in  the  natural  setting,  the  breadth  of  respondents  will  be  valued  and  the  choices  will  be  made  to  utilize  cheaper 
methods  than  traveling  to  the  site.  It  is  the  same  principle  as  we  use  in  the  assignation  of  media  in  instruction.  We 
ought  not  use  a media  merely  because  it  is  available,  new,  or  glitzy~or  well  funded,  but  rather  because  it  is  what  is 
called  for  in  the  instructional  design  based  on  the  assessment,  strategy  selection,  goals  and  objectives  for  the  learner. 
In  this  same  way,  we  should  select  telephone  or  CMI  options  for  collecting  qualitative  data  because  it  is  supported 
by  the  research  question . Methodological  choices  should  not  be  made  for  financial  reasons  alone  any  more  than 
they  should  be  made  for  convenience  reasons  alone— thus  the  negative  perception  of  samples  of  convenience  studies. 
Instead  we  should  allow  our  research  question  to  drive  our  interview  method  choices  among  face-to-face,  telephone, 
and  CMI. 

Observations 

Qualitative  observations  (Adler  & Adler,  1994)  elicit  some  of  the  same  issues  as  interviews,  but  the 
concerns  here  are  even  more  pronounced.  How  is  it  possible  to  "observe"  an  online  class?  We  know  what  it  means 
to  observe  a face-to-face  classroom,  and  we  know  how  to  design  an  observation  instrument  to  assist  us  in  focusing 
on  the  proper  interactions,  experiences,  cultures,  and  environmental  cues.  However,  we  don’t  know  what  any  of  this 
means  in  the  online  environment.  Do  we  observe  individuals  at  their  machines  in  their  home  space?  Do  we  observe 
the  class  as  it  interacts  online?  Is  the  electronic  space  the  actual  classroom?  What  sorts  of  things  are  we  looking  for, 
what  type  of  instrument  will  help  us  to  focus  on  those  things?  Participant  observations  where  the  researcher  is  part 
of  the  culture  in  which  the  observation  is  being  conducted,  as  is  the  case  with  ethnographic  studies,  leave  the  role  of 
the  researcher  even  more  undefined  in  online  environments.  It  is  important  to  consider  the  practical  as  well  as  the 
theoretical  and  philosophical  aspects  of  participant  observations  in  online  class  settings. 

In  our  experiences  this  is  perhaps  one  of  the  most  difficult  things  to  handle  in  online  learning  environment 
inquiry.  Because  we  do  have  certain  "scripts"  (mental  or  physical)  which  guide  our  interview  protocols,  the 
interview  procedures  are  still  somewhat  familiar  to  us  in  electronic  media.  Although  the  social  nature  of  the 
interview  is  significantly  altered,  the  pragmatics  of  how  to  go  about  it  are  still  relatively  similar-that  is  to  set  out  as 
series  of  questions  (structured,  unstructured,  semi-structured)  and  begin  conversing  with  respondents.  But  how  do 
we  go  about  really  observing  the  online  learning  environment?  Is  it  even  possible?  Here  is  an  even  more  evident 
exposure  of  the  social  nature  of  qualitative  inquiry  (Wolcott,  1995).  Here  we  have  a real  rub,  what  sort  of 
observational  techniques  should  we  use,  can  we  rely  on  the  old  checklists  for  observations  that  we  had  used 
traditionally?  Do  we  have  to  shift  foci  in  the  same  ways  that  we  do  in  traditional  settings?  Unfortunately,  our 
experience  raised  more  questions  than  answers  in  this  regard.  We  attempted  to  "observe"  online  settings  by  reading 
through  the  ongoing  synchronous  exchanges  such  as  chats  and  asynchronous  discussion  forum  communication.  In 


4 


13 


the  end,  our  experience  was  so  limited  in  this  regard,  that  there  is  little  we  can  really  say  by  way  of  overall 
impressions  or  guidance.  Rather  we  can  merely  raise  these  questions  for  careful  —even  cautious—  consideration  by 
those  embarking  in  online  learning  environment  inquiry. 


Document  Analysis 

Document  analysis  (Hodder,  1994)  is  perhaps  the  least  difficult  qualitative  data  issue  to  be  resolved.  Since 
there  are  fewer  abstractions  that  are  important  (e.g.,  voice  tone,  culture  of  a classroom,  etc.)  in  document  analysis, 
the  issues  are  a little  less  complicated.  Clearly  online  course  materials  as  with  face-to-face  course  materials  are 
prime  candidates  for  document  analysis  in  such  studies.  In  the  same  way,  student  materials  are  also  documents.  But 
in  the  online  environment,  there  are  typically  many  more  "printed”  resources  than  there  may  be  in  the  face-to-face 
environment.  What  is  to  be  considered  a document?  Is  the  chat  that  the  students  conduct  on  regular  Monday 
meeting  times,  for  example,  to  be  considered  documentary,  observation,  or  interview  data?  Documents  typically 
included  in  qualitative  research  for  data  analysis  appear  in  the  form  of  written  text.  Written  text  assumes  a purpose 
depending  on  the  author  and  the  intended  use  of  the  document.  Creswell  (1998)  distinguishes  between  texts  written 
as  records  of  information,  i.e.,  public  documents,  reports,  contracts  and  others  such  as  diaries  or  journals,  written  for 
personal  use.  Journals,  logs  and  diaries  are  sometimes  requested  by  the  researcher  as  a way  for  participants  to  keep 
track  of  and  reflect  on  their  activities  in  a particular  situation  or  context. 

In  online  learning  environments,  information  is  presented  in  the  form  of  written  text  in  several  aspects 
including  (but  not  limited  to)  synchronous  chat,  discussion  forum,  project  submissions,  email  communication  and 
written  reports  or  summaries.  As  these  class  activities  are  utilized  in  ways  that  intend  to  mirror  activities  typically 
integral  to  face-to-face  learning  environments,  in  which  case  they  would  not  be  considered  in  document  analysis,  but 
rather  as  observation  or  interview  data.  In  what  ways  does  this  change  the  nature  of  data  collected.  In  the  case  of 
document  analysis,  wherever  this  cautious  distinction  is  drawn  and  there  is  care  to  maintain  authenticity  in  the 
documents  themselves,  it  is  likely  that,  in  this  one  area,  there  is  the  least  impact  of  the  switch  from  face-to-face  to 
online  inquiry  subject.  This  is  primarily  because  there  is  inherently  less  "socialness"  to  the  procedures  within  most 
document  analyses  even  within  the  qualitative  tradition.  It  is  more  anthropological  in  nature,  uncovering  the 
cultures  and  understandings  of  authors  as  a result  of  careful  dissection  of  documented  data. 

Conclusion  and  a Note  of  Caution 

With  the  growing  interest  in  online  education  and  the  vast  difference  in  teaching  and  learning  between 
online  and  face-to-face  learning  environments,  it  is  becoming  more  and  more  necessary  to  gain  a better 
understanding  of  the  experiences  students  face  in  new  distance  education  endeavors  of  many  institutions  of  higher 
education.  While  there  is  little  research  available  in  this  area,  we  believe  qualitative  methods  in  seeking  this 
information,  are  not  only  appropriate  but  necessary  in  developing  a greater  understanding  of  student  experiences. 
However,  the  employ  of  qualitative  methods  founded  on  assumptions  of  the  social  nature  of  inquiry  can  only  be 
used  with  great  caution.  It  is  essential  that  as  we  continue  to  examine  online  learning  environments  in  qualitative 
ways  that  we  carefully  reflect  on  the  experience  of  doing  online  interviews,  online  observations  and  electronic 
document  analysis  and  try  to  capture  how  we  see  these  experiences  as  differing  from  previous  experiences  in 
qualitative  research. 

It  is  essential  that  all  good  qualitative  research  have  certain  hallmarks  of  quality,  and  in  one  final  note,  we 
wish  to  emphasize  the  importance  of  disclosing  one's  researcher  identity.  Even  more  so  than  in  many  other  areas, 
the  online  revolution  has  recently  turned  into  an  online  opinion  war  with  many  folks  lining  up  on  both  sides,  pro 
(Dede,  1997  ) and  con  (Noble,  1998).  Because  the  online  learning  craze  is  likely  to  affect  our  own  lives  and 
livelihoods,  it  is  essential  that  we  are  clear  with  all  readers  of  precisely  our  own  feelings  and  biases  with  regard  to 
this  new  innovation  in  honest,  up  front  ways. 
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Abstract 

Recent  technological  advances  now  make  possible  full  integration  of  sound  in  instructional  software.  This 
paper  explores  the  systematic  use  of  sound  to  enhance  learning.  It  illustrates  a model  of  the  “ instructional 
communication  system  " based  on  a combination  of  information-processing  and  communication  theories.  The  model 
identifies  and  characterizes  the  way  sound  might  be  used  to  optimize  learning  from  multimedia  instruction. 

Technological  and  cost  barriers  to  full  integration  of  sound  in  instructional  software  have  recently 
disappeared.  Conceptual  and  preconceptual  barriers,  however,  still  appear  to  prevent  software  designers  from  using 
sound  more  effectively  in  their  instructional  products.  Interface  books  seldom  discuss  the  use  of  sound,  and  when 
they  do,  the  use  most  often  discussed  is  simple  verbatim  narration  of  on-screen  text  (see  Bickford,  1997;  Cooper, 
1995;  Galitz,  1997;  Mandel,  1997).  Because  most  “classics’*  of  instructional  interface  design  were  written  before 
sound  was  a viable  design  component,  sound  is  seldom  well  discussed  (see  Alessi  & Trolip,  1985,  1991;  Hannafin  & 
Peck,  1988;  Jonassen,  1988;  Keller,  1987).  In  general,  interface  design  guidelines  identify  three  main  uses  of  sound 
in  instructional  software:  to  alert  learners  to  errors,  to  provide  stand-alone  examples  (like  musical  passages  or 
digitized  versions  of  speeches),  or  to  narrate  text  on  the  screen  (for  redundant  presentation,  for  non-readers,  or  for 
those  with  auditory  limitations).  Review  of  research  on  sound  in  instructional  software  reveals  a focus  on  the  third 
use  cited  above,  digitized  or  computer-generated  synthetic  speech  narration  (see  Barron  & Atkins,  1994;  Mann, 
1995;  Shih  & Alessi,  1996).  While  some  outside  education  have  considered  non-speech  interface  sounds  (Blattner, 
Sumikawa,  & Greenberg,  1989;  Gaver,  1986),  many  promising  uses  remain  unexplored. 

Before  one  can  determine  sound’s  potential  contribution  to  instructional  software,  however,  one  must  have 
a clear  picture  of  the  component  processes  involved  in  learning.  Therefore,  this  paper  begins  by  exploring  the 
operations  and  limiting  factors  of  information-processing. 

Information-processing  Operations  and  Limiting  Factors 

According  to  information-processing  theory,  learning  emerges  from  processing  interactions  among 
information  from  the  environment  and  the  learner’s  knowledge  and  previous  experiences.  Most  theorists  have 
adopted  at  least  the  basic  structure  of  the  three-stage  memory  model  first  proposed  by  Atkinson  and  Shiffrin  in  1968. 

The  Atkinson-Shiffrin  Information-processing  Model 

In  the  Atkinson-Shiffrin  model,  environmental  stimuli  in  their  primitive  form  are  first  handled  by  a sensory 
information  store,  or  sensory  register.  Signals  held  here  are  readily  displaced  by  subsequent  signals  in  the  same 
sensory  channel.  The  sensory  register  filters  and  then  routes  incoming  signals  to  a second,  short-term  store  where 
information  is  held  temporarily  until  it  can  be  encoded  for  storage.  Encoding  is  the  process  of  building  relationships 
and  connections  within  new  material  or  between  new  material  and  existing  knowledge  structures.  Once  encoded,  the 
information  is  moved  into  long-term  store  in  the  form  of  images  (the  autobiographical  knowledge  or  episodic 
memories  one  has  for  things  that  have  been  personally  experienced)  or  schemas  (the  organized,  propositional 
knowledge  one  has  for  the  meanings,  rules,  and  algorithms  used  to  manipulate  and  understand  the  many  symbol 
systems  encountered  in  life)  (Tulving,  1972).  Long-term  store  is  both  the  place  where  we  hold  newly  encoded 
information  and  the  place  from  which  we  retrieve  well-established  memories.  Recovering  information  from  long- 
term store  requires  cues  that  may  be  supplied  externally  by  the  situation  or  internally  by  one’s  existing  memories. 
These  cues  are  used  to  search  long-term  store  in  order  to  identify  and  retrieve  matches.  Control  processes  “oversee” 
the  cognitive  system  by  regulating  the  exchange  of  information  between  the  sensory  register  and  long-term  store, 
determining  which  search-and-retrieval  strategies  to  use  to  access  information  from  long-term  store,  and  deciding 
when  sufficient  information  has  been  retrieved. 

Information-processing  theorists  maintain  that  learning  occurs  when  information  that  has  been  transferred 
to  and  stored  in  long-term  memory  can  be  retrieved  when  needed.  Transforming  incoming  environmental  stimuli 
into  learned  images  and  schema  involves  three  main  operations:  acquisition,  processing,  and  retrieval.  It  appears, 
however,  that  limitations  in  each  of  these  operations  may  restrict  the  amount  of  data  one  can  store  long-term. 
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Limiting  Factors  in  Information  Processing 

In  order  to  acquire  or  make  sense  of  the  constant  barrage  of  sensory  information,  an  individual  must  decide, 
often  unconsciously,  which  information  to  attend  to  and  which  to  ignore.  To  explain  this  phenomenon,  Broadbent 
(1958)  posited  that  all  information  reaching  the  sensory  register  is  subjected  simultaneously,  or  in  parallel,  to  a 
preliminary  analysis  based  on  prior  knowledge.  From  this  pre-perceptual  analysis  of  the  entire  sensory  scene,  one 
chooses  a smaller  subset  of  stimuli  to  process  successively,  or  in  serial,  through  the  rest  of  the  cognitive  system.  The 
“bottleneck”  created  between  parallel  pre-perceptual  and  serial  perceptual  stages  restricts  the  amount  of  information 
entering  the  cognitive  system.  Individuals  remain  essentially  unaware  of  information  not  selected  for  attention. 

Like  many  later  researchers,  Wundt  (1896/1897)  found  short-term  store  is  also  of  limited  capacity:  There  is 
a limit  to  the  amount  of  information,  or  maximal  cognitive  load , an  individual  can  process  in  short-term  store  at  any 
given  time.  Although  it  may  be  that  cognitive  load  varies  somewhat,  depending  upon  the  nature  of  input  stimuli,  our 
capacity  for  processing  incoming  data  is  certainly  limited  to  some  finite  quantity.  Information  that  exceeds  cognitive 
processing  capacity  is  dropped  from  short-term  store  without  being  processed.  Further,  unless  information  that 
enters  the  store  is  rehearsed,  it  decays  within  approximately  five  to  twenty  seconds.  Short-term  store  limitations 
dictate  that  data  not  encoded  and  moved  into  long-term  store  must  be  overwritten  to  make  room  for  new  incoming 
stimuli  (as  when  we  forget  a new  phone  number  after  hearing  another  series  of  numbers)  or  consciously  rehearsed 
and  then  discarded  immediately  after  use  (as  when  we  repeat  a telephone  number  aloud  until  we  have  dialed  it). 
Memories  often  seem  to  fade  with  the  passage  of  time.  Forgetting  is  a failure  to  retrieve  information  from  long-term 
store.  There  are  three  general  hypotheses  about  the  factors  that  cause  forgetting,  each  of  which  probably  contributes 
to  overall  retrieval  problems.  The  decay  hypothesis  asserts  that  the  strength  of  a memory  simply  weakens  over  time 
and  therefore  is  harder  to  retrieve  (Wickelgren,  1976).  The  interference  hypothesis  claims  that  competition  among 
memories  blocks  the  retrieval  of  a target  memory  (Postman,  1961).  The  retrieval-cue  hypothesis  asserts  that  at  the 
time  of  retrieval  we  lose  access  to  the  internal  “indices”  that  point  to  the  memory’s  location  in  long-term  store 
(Norman,  1982).  There  is  some  evidence  to  suggest  that  once  information  has  been  moved  to  long-term  store,  it 
remains  there  forever  (Nelson,  1971).  While  this  means  memories  may  never  actually  leave  long-term  store, 
individuals  certainly  can  lose  access  to  them. 

Berio  (1960)  suggested  that  the  study  of  learning  processes  and  the  study  of  communication  processes 
differ  only  in  their  point  of  view.  While  learning  models  generally  begin  with  and  focus  on  how  messages  are 
received  and  processed  by  learners,  communication  models  most  often  begin  with  and  focus  on  how  messages  are 
sent.  Learning  from  instructional  software,  therefore,  might  be  viewed  as  an  instructional  communication  system 
with  a set  of  interrelated  parts  working  together  to  produce  learning  (Banathy,  1996). 

Communication  Operations  and  Limitations 

Communication  is  the  transmission,  reception,  and  decoding  of  signals.  As  was  the  case  with  information- 
processing theory,  one  model  — Shannon  and  Weaver’s  The  Mathematical  Theory  of  Communication  (1949/1969) — 
appears  to  have  been  particularly  influential  in  shaping  communication  theory. 

The  Shannon-Weaver  Communication  Model 

The  Shannon-Weaver  model  proposes  that  all  communication  processes  begin  when  a source , desiring  to 
produce  some  outcome,  chooses  a message  to  be  communicated.  The  message  is  encoded  to  produce  a signal 
appropriate  for  transmission  over  the  channel  that  will  be  used.  After  the  message  has  been  transmitted,  a receiver 
then  decodes  the  message  from  the  signal  transmitted.  All  channels  have  limited  capacity.  In  humans,  channel 
capacity  generally  refers  to  the  physiological  and  psychological  limitations  on  the  number  of  symbols  or  stimuli  that 
individuals  can  process.  When  more  symbols  are  transmitted  than  a channel  can  handle,  some  information  is  lost. 

According  to  Shannon  and  Weaver,  communication  is  “perfect”  when  the  information  contained  in  a 
message  affects  the  receiver  in  exactly  the  way  intended  by  the  source.  Communication  is  rarely  perfect,  however;  at 
any  point  things  can  get  added  to  the  signal  that  were  not  originally  intended  by  the  source.  This  spurious 
information,  or  noise , introduces  errors  that  increase  the  uncertainty  in  the  situation  and  make  the  signal  harder  for 
the  receiver  to  reconstruct  accurately. 

Limiting  Factors  in  Communication 

Shannon  and  Weaver  divided  the  analysis  of  communication  problems  into  three  levels.  “Level  A”  deals 
with  how  accurately  the  signal  is  received.  When  competing  external  or  internal  stimuli  exist  in  a communication 
channel,  the  resulting  noise  introduces  technical  errors  that  can  overpower  all  or  part  of  a signal  transmission.  This 
disruption  prevents  the  receiver  from  being  able  to  select  the  communicated  signal  for  decoding.  No  matter  how 
accurately  a message  is  transmitted,  however,  if  it  cannot  be  decoded  by  the  receiver  it  is  not  likely  to  convey  the 
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intended  message.  “Level  B,”  therefore,  concerns  how  precisely  the  received  signal  conveys  the  intended  message. 
Decoding  requires  the  receiver  to  analyze  an  incoming  signal  based  on  his  or  her  existing  schemas.  When  no 
interpretive  framework  exists  and  none  is  supplied  by  the  source,  the  resulting  noise  introduces  semantic  errors  that 
prevent  the  signal  from  conveying  the  intended  message.  Even  when  a message  is  interpreted  correctly,  it  still  may 
not  accomplish  the  source’s  goal.  Thus,  “Level  C”  involves  whether  the  received  message  ultimately  produces  the 
outcome  desired  by  the  source.  To  effect  an  outcome,  the  elements  and  structure  of  the  message  that  assign 
connotative  meaning  — such  as  aesthetic  appeal,  style,  execution,  and  other  psychological  and  emotional  factors — 
must  mesh  with  the  receiver’s  own  relevant  beliefs,  cultural  values,  and  experiences.  If  this  synthesis  leads  the 
receiver  to  make  inferences  about  the  message  that  are  not  intended  by  its  source,  the  resulting  noise  introduces 
conceptual  errors  that  can  prevent  the  communication  from  producing  the  desired  result. 

Although  Shannon  and  Weaver  confined  their  work  primarily  to  the  study  of  Level  A problems  of 
mechanistic  communication  systems,  they  contended  that  improving  the  effectiveness  and  efficiency  of  the 
communication  process  overall  requires  applying  concepts  from  their  model  to  all  three  levels  of  communication 
problems.  Their  work  suggested  that  there  may  be  ways  to  anticipate  communication  difficulties  and  “front  load” 
messages  with  the  cues  necessary  to  “squelch”  noise  even  before  it  occurs. 

Using  Sound  to  Squelch  Noise  in  Instructional  Communication  Systems 

Sounds  can  support  learning  in  a variety  of  ways.  They  can  gain  and  focus  learners’  attention,  reducing  the 
distraction  of  competing  stimuli,  engaging  interest  over  time,  and  making  environments  more  tangible  and 
emotionally  arousing  (Kohfeld,  1971;  Bernstein,  Clark,  & Edelstein,  1969;  Thomas  & Johnston,  1984).  They  can 
help  learners  condense,  elaborate  upon,  and  organize  details  about  their  surroundings,  helping  them  to  see 
interconnections  among  new  pieces  of  information  (McAdams,  1993;  Winn,  1993;  Yost,  1993).  They  can  provide  a 
familiar  setting  that  may  help  learners  relate  incoming  information  to  existing  knowledge  (Deutsch,  1980,  1986; 
Gaver,  1993).  Thus,  McAdams  and  Bigand  (1993)  argued  that  sound  is  uniquely  suited  to  assist  in  the  acquisition, 
processing,  and  retrieval  of  new  information  for  those  who  are  not  hearing-impaired.  If  this  is  true,  there  may  be 
systematic  ways  to  design  multi-cue  instructional  messages  that  overcome  information-processing  noise  and 
optimize  learning. 

The  Role  of  Multi-cue  Messages  in  Instructional  Communication  Systems 

For  some  time  it  has  been  thought  that  simply  adding  cues  to  messages  might  improve  the  effectiveness  of 
instructional  communication.  The  idea  behind  cue  summation  is  that  the  more  cues  used,  whether  within  or  across 
sensory  channels,  the  greater  the  amount  of  information  communicated  and  the  more  learning  gained.  While  the 
results  of  cue  summation  studies  appear  contradictory  on  the  surface,  Severin  (1967)  maintained  the  differences 
might  be  explained  by  the  degree  of  redundancy  among  cues  used  in  the  treatments.  Severin  noted  that  studies  that 
found  no  difference  between  multiple-cue  and  single-cue  communication  used  cues  that  were  almost  totally 
redundant,  such  as  text  coupled  with  word-for-word  narration.  In  these  studies  the  wedded  cues  apparently  neither 
competed  with  each  other  nor  supplied  new  information.  In  contrast,  studies  that  found  multiple-cue 
communications  less  effective  than  single-cue  communications  used  cues  with  no  redundancy  between  them,  such 
as  text  coupled  with  unrelated  speech.  In  these  studies,  the  dueling  cues  probably  exceeded  channel  capacity, 
producing  noise  that  decreased  communication  efficiency.  Severin  concluded  that  studies  that  found  multi-cue 
communications  to  be  more  effective  than  single-cue  communications  used  cues  that  were  partially  redundant,  like 
pictures  coupled  with  related  narration.  In  these  studies,  primary  and  secondary  cues  appear  offset  just  enough  for 
the  secondary  cue  to  supply  the  right  balance  of  redundancy  and  new  information.  Severin  contended  that  multi-cue 
messages  can  be  designed  to  help  improve  instructional  communication.  The  question  is  not  just  whether  the 
message  contains  multiple  cues,  but  whether  those  secondary  cues  supply  the  optimal  amount  of  redundancy. 

The  Role  of  Redundancy  in  Instructional  Communication  Systems 

Redundancy  is  the  information  message  cues  share:  the  parts  that  “overlap.”  For  example,  a source  might 
attempt  to  correct  technical  problems  in  the  system  (Level  A)  by  retransmitting  or  amplifying  the  signal.  This 
content  redundancy  often  can  help  overcome  transmission  errors  by  completing  obstructed  signals  or  by  preventing 
the  interference  in  the  first  place.  A source  anticipating  semantic  problems  in  the  system  (Level  B)  might  attempt  to 
correct  them  by  supplying  the  relevant  connections  between  and  among  related  message  signals.  This  context 
redundancy  often  can  help  overcome  misinterpretations  by  furnishing  denotative  meanings  for  signals.  A source 
might  attempt  to  correct  conceptual  problems  in  the  system  (Level  C)  by  carefully  choosing  signals  that  make 
appropriate  links  to  receivers’  preexisting  concepts  in  memory.  This  construct  redundancy  clarifies  the  connotative 
meanings  behind  message  signals  and  reduces  misunderstandings. 

When  a source  anticipates  noise  at  the  various  levels  of  communication,  the  trick  may  be  in  knowing  how 
much  and  which  sort  of  between-cue  message  redundancy  to  include  in  order  to  counteract  noise.  Striking  this 
optimal  balance  may  also  be  the  key  to  successful  instructional  communication.  Further,  it  appears  that  multi-cue 
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instructional  messages  incorporating  sound  might  both  deliver  sufficient  amounts  of  new  information  and  supply  the 
noise-defeating  content,  context,  and  construct  redundancy  necessary  to  enhance  learning.  Theories  of  system 
optimization,  such  as  that  presented  by  Wilde  and  Beightler  (1967),  recommend  creating  a model  of  the  system  in 
order  to  understand  precisely  what  the  system  must  do  to  accomplish  its  goal.  This  model  might  then  serve  as  a 
frame  of  reference  for  subsequent  evaluation  of  the  system. 

Modeling  the  Instructional  Communication  System 

Understanding  the  underlying  component  processes  of  instructional  communication  might  begin  by  adding 
the  receiver’s  information-processing  transactions  to  the  Shannon-Weaver  model.  Figure  1 depicts  the  receivers’ 
component  processes  in  more  detail,  illustrating  an  idealized  representation  of  the  three  levels  of  communication  as 
three  learning  phases  (select,  analyze,  and  synthesize)  and  depicting  instructional  communication  limitations  — 
channel  capacity  and  noise — as  constraints  on  the  system. 

If  we  were  trying  to  discover  only  what  output  the  system  produces  for  a given  mixture  and  amount  of 
input,  this  “black  box”  model  might  be  adequate.  However,  our  concern  here  is  to  determine  what  amounts  and 
mixtures  of  input  at  each  phase  of  learning  will  bring  about  the  kinds  of  system  activity  necessary  to  produce 
optimal  learning  output  (Churchman,  1968).  In  order  for  us  to  understand  how  input  might  affect  instructional 
communication,  the  model  must  approximate  more  closely  the  complex  circles  of  influence  that  exist  among  the 
proposed  learning  phases,  the  system’s  component  information-processing  operations,  and  the  human- 
communication  system  constraints.  Figure  1 neither  depicts  the  role  that  acquisition,  processing,  and  retrieval 
operations  play  in  each  learning  phase  nor  suggests  how  the  system  might  utilize  the  content,  context,  and  construct 
redundancy  in  an  instructional  message  to  help  the  receiver  overcome  system  constraints. 

In  order  to  explore  the  area  inside  the  boxes,  it  might  be  useful  to  move  from  the  traditional  flow-chart  type 
of  diagram  toward  a structural  illustration  that  approximates  more  closely  the  more  dynamic  nature  of  the 
instructional  communication  system.  The  model  illustrated  in  Figure  2 and  in  subsequent  figures  in  this  chapter 
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draws  parallels  between  the  instructional  communication 
system  and  the  process  of  crystal  formation  in  chemistry. 

Thus,  Figure  2 eliminates  the  traditional  boxes  and 
arrows  in  favor  of  a grid  of  intersecting  lines  that  represents 
learning  phases  and  operations  as  an  instructional  message 
moves  through  the  instructional  communication  channel.  The 
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Figure  2.  A representational 
illustration  of  the  instructional 
2 Communication  system . 


channel  is  drawn  as  the  area  between  two  parallel  lines  running  vertically  from  the  top  to  the  bottom  of  the  diagram. 
The  message  is  represented  as  a set  of  increasingly  lighter  concentric  circles  (moving  from  inner  core  to  outer 
circle).  The  areas  between  the  lines  running  horizontally  from  left  to  right  represent  the  three  transactions  or 
“phases”  of  learning  — selection,  analysis,  and  synthesis — identified  for  use  in  the  earlier,  linear  model.  This  is  the 
learner’s  working  memory.  The  areas  between  the  channel  lines  running  diagonally  (from  lower-left  to  upper-right) 
represent  the  three  information-processing  operations  that  the  receiver  uses  at  each  phase  of  learning  in  order  to 
process  the  instructional  signal. 

The  dividing  lines  between  acquisition,  processing,  and  retrieval  dissect  the  channel  diagonally  in  order  to 
illustrate  how  each  operation  is  applied  in  varying  amounts  at  each  learning  phase.  Processing  is  depicted  as  the 
middle  of  the  three  because  it  relies  on  acquisition  and  retrieval  to  supply  the  information  and  memories  it  acts  upon. 
During  selection,  processing  calls  upon  acquisition  heavily;  in  contrast,  only  the  most  salient  memories  are  retrieved 
during  selection.  During  analysis,  processing  is  central  — although  acquisition  and  retrieval  are  also  relatively  active. 
During  synthesis,  processing  calls  upon  retrieval  most  heavily,  while  only  the  most  salient  new  stimuli  are  acquired. 

The  area  at  the  bottom  of  the  model  represents  long-term  store,  with  its  schema  particles  “suspended”  and 
“fluid.”  These  schemas  are  retrieved  from  memory  to  help  learners  make  sense  of  new  information.  The  particles  in 
the  long-term  store  suspension  are  not  disordered;  related  crystals  naturally  gravitate  toward  one  another  and  may 
align  to  form  more  complex  and  interdependent  structures.  The  extent  to  which  particles  remain  suspended  and 
fluid,  however,  depends  upon  how  often  they  are  “agitated.”  Particles  that  have  just  recently  been  stored  or  retrieved 
from  memory  waft  to  the  top  of  long-term  store,  carrying  others  around  them  along  in  their  wake.  This  activity 
makes  these  memories  easily  accessible  to  working  memory  and  likely  to  be  among  the  first  retrieved  in  later 
learning  situations  (Norman,  1982).  Schema  particles  suspended  in  long-term  memory  will  settle  slowly  if  left 
stagnate.  Unless  schemas  occasionally  are  retrieved  from  long-term  store  and  the  “silt”  stirred  up,  they  become  too 

deeply  embedded  and  are  forgotten.  Stagnate 
memories  lack  the  fluidity  to  be  applied  to  new 
learning  situations. 

Because  of  the  inevitable  differences  among 
learners’  characteristic  cognitive,  conative,  and 
affective  traits,  the  instructional  communication 
channel  is  always  prone  to  noise  (Como  & Snow, 
1986).  For  example,  the  same  instructional  method 
that  gains  one  learner’s  attention  may  not  gain 
another’s  (Biehler  & Snowman,  1982).  Some  learners 
will  lack  the  prior  knowledge  they  need  to  help  them 
make  sense  of  the  new  information  (Rumelhart  & 
Norman,  1981).  Still  other  learners  may  be  unable  or 
unwilling  to  apply  their  knowledge  mindfully  when 
appropriate  (Langer,  1989).  The  potential  for 
acquisition,  processing,  and  retrieval  noise  in  the 
selection  phase  is  depicted  in  Figure  3 as  “NA,”  “NP,” 
and  tcNR,”  respectively.  (Despite  possible  similarities 
between  the  letter  code  representations  used  in  the 
model  and  the  names  of  actual  chemical  elements, 


"NA,"  “NP,"  and 
“NR"  symbolize  the 
potential  for  noise  to 
develop. 

Some  learners  will 
have  more  potential 
for  learning 
problems  than 
others  (due  to 
cognitive,  conative, 
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At  each  phase, 
the  potential  for 
noise  increases, 
as  well  as  the 
consequences 
of  that  noise. 


Figure  3 . Redundant  message  enters  channel  filled  with 
the  potential  for  developing  noise. 

such  letter  codes  here  do  not  represent  such 
elements.) 

At  each  deeper  phase  of  learning,  the 
relative  “strength”  of  potential  noise  as  well  as  the 
ultimate  consequences  of  that  noise  increase  by  some 
factor.  For  example,  while  it  can  be  difficult  to 
overcome  acquisition  noise  at  selection  in  order  to 
gain  a learner’s  attention,  it  often  is  much  harder  to 
overcome  acquisition  noise  in  analysis  in  order  to 
focus  a learner’s  attention,  and  harder  still  to  hold  a 
learner’s  attention  over  time  for  synthesis.  While 
there  is  no  way  to  know  the  actual 
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Figure  4.  Message  contains  three  layers  of  “solute  ” with  varying 
amounts  of  content,  context,  and  construct  redundancy  “ molecules . ” 
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proportions/concentrations  of  noise  at  each  phase,  for  purposes  of  illustration,  the  authors  depicted  the  potential  for 
acquisition,  processing,  and  retrieval  noise  as  “NA,”  “NP,”  and  “NR”  in  the  selection  phase,  “2NA,”  “2NP,”  and 
“2NR”  in  the  analysis  phase  and  as  “4NA,”  “4NP,”  and  “4NR”  in  the  synthesis  phase.  Similarly,  because  the 
proportion  of  acquisition,  processing,  and  retrieval  noise  potential  is  likely  to  be  closely  related  to  the  proportion  of 
acquisition,  processing,  and  retrieval  involved  at  each  learning  phase,  acquisition  noise  potential  at  selection  is 
depicted  using  4 “units”  as  compared  with  1 “unit”  of  acquisition  noise  potential  at  synthesis.  Conversely,  retrieval 
at  selection  is  depicted  as  fairly  low  (1  “unit”)  whereas  it  is  comparatively  high  at  synthesis  (4  “units”).  The 
potential  for  processing  noise  remains  the  same  at  each  learning  phase  (4  “units”). 

Figure  4 supplies  a closer  look  at  the  instructional  message.  The  black  circle  labeled  “CORE”  represents 
the  entropic,  core  knowledge  that  the  instructional  message  is  intended  to  convey.  The  increasingly  lighter  circles 
around  the  core  (moving  from  inside  to  outside)  represent  its  layers  of  redundancy  that  have  been  “formulated”  for 
each  learning  phase.  Figure  6 depicts  the  ratio  of  content,  context,  and  construct  redundancy  “molecules”  that  would 
be  appropriate  for  the  learning  situation  depicted  in  Figure  3.  That  means  the  number  of  redundancy  molecules 
within  each  ring  are  balanced  for  the  information-processing  needs  of  its  intended  learning  phase  and  the  potential  Cl 
for  channel  noise  given  the  audience  and  the  content.  Recall,  however,  that  the  proportions/concentrations  used  here 
are  merely  for  illustration. 

As  depicted  in  Figure  5,  when  an  instructional  message  enters  the  selection  phase  of  working  memory  its 
first  layer  of  redundancy  dissolves.  Depending  upon  the  learner’s  ability  or  willingness  to  exert  information- 


Figure  5.  First  redundancy  layer 
dissolves  stimulating  selection- 
level  information-processing . 


Figure  6 . Information-processing  Figure  7.  Noise  potential  is 

effort  “reacts  ” with  potential  noise  “ neutralized . " 

and  the  redundancy  in  the  message. 
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processing  effort,  the  content,  context,  and  construct  solute  stimulates  the  production  of  acquisition,  processing,  and 
retrieval  ions  (“EA,”  “EP,”  and  “ER,”  respectively).  When  these  ions  are  added  to  a solution  that  already  contains 
redundancy  dissolved  from  the  message  (Cn,  Cx,  and  Cs),  and  various  forms  of  information-processing  noise 
potential  (NA,  NP,  and  NR),  three  “chemical  reactions”  occur  (Figure  6).  These  reactions  theoretically  could  be 
depicted  mathematically: 

NA  + Cn  + EA  ->  A2Cn  + NE 
NP  + Cx  + EP  ->  P2Cx  + NE 
NR  + Cs  + ER  ->  R2Cs  + NE 

The  first  formula  means  that  when  the  potential  for  acquisition  noise  is  combined  with  content  redundancy  and  the 
learner’s  effort  to  acquire  the  material,  the  resulting  products  are  gained  attention  (A2Cn)  and  residual  positive 
affects  of  learning,  such  as  improved  attitudes  and  feelings  of  success,  that  serve  as  catalysts  for  continued 
leaming(NE,  as  in  “noise  + energy”).  The  second  formula  indicates  that  when  the  potential  for  processing  noise  is 
combined  with  context  redundancy  and  the  learner’s  effort  to  process  the  material,  the  resulting  products  are  isolated 
relevant  stimuli  (P2Cx)  and  more  positive  residual  (NE).  The  third  formula  indicates  that  when  the  potential  for 
retrieval  noise  is  combined  with  construct  redundancy  and  the  learner’s  effort  to  retrieve  schemas  from  long-term 
store,  the  resulting  products  are  retrieval  of  the  appropriate  constructs  from  memory  (P2Cx)  and  still  more  NE.  The 
final  outcome  of  successful  selection-level  processing  is  depicted  in  Figure  7.  At  this  point  in  the  instructional 
communication  system,  the  learner  is  interested  in  the  message  and  has  selected  it  for  further  processing.  Further, 
higher  concentrations  of  NE  make  the  working  memory  solution  thicker,  slowing  the  message  and  allowing  deeper 
processing. 

Similar  reactions  occur  in  the  analysis  phase.  Once  again,  using  our  hypothetical 
proportions/concentrations,  the  analysis  reactions  might  be  depicted  like  this: 

2NA  + Cn2  + EA  ->  A4Cn2  + 2NE 
2NP  + Cx2  + EP  ->  P4Cx2  + 2NE 
2NR  + Cs2  + ER  ->  R4Cs2  + 2NE 

Analysis-level  reactions  yield  focused  attention  (A4Cn2),  information  organizing  and  categorizing  (P4Cx2),  efforts  to 
build  upon  existing  knowledge  (R4Cs2)  and  even  higher  levels  of  positive  residual  (2NE).  At  this  point  in  the 
instructional  communication  system,  the  learner  is  curious  about  the  message  and  is  actively  analyzing  its  meaning. 
Further,  higher  concentrations  of  NE  continue  to  make  the  working  memory  solution  thicker  and  decrease  message 
speed. 

Finally,  the  synthesis  reactions  employing  our  hypothetical  proportions/  concentrations,  might  look  like 

this: 

4NA  + C114  + 4EA  ->  AgCo*  + 4NE 
4NP  + CX4  + 4EP  ->  P8Cx4  + 4NE 
4NR  + Cs4  + 4ER  ->  R8Cs4  + 4NE 

The  product  of  synthesis-level  reactions  is  attention  held  over  time  (A8Cn4),  elaboration  upon  the  new  information 
(P0Cx4),  construction  of  more  transferable  knowledge  structures  (R8Cs4),  and  still  higher  levels  of  positive  residual 
(2NE).  At  this  point  in  the  instructional  communication  system,  the  learner  is  engaged  in  the  message  and  is  affected 
by  its  larger  meaning.  Still  higher  concentrations  of  NE  continue  to  “thicken”  the  solute  and  slow  the  message’s 
passage  through  the  system,  fostering  deeper  processing. 

Thus,  as  each  layer  of  carefully  chosen  redundant  material  dissolves,  the  message  “solute”  acts  to 
“neutralize”  the  acquisition,  processing,  and  retrieval  noise  in  that  phase.  In  Figures  5-7,  the  first  layer  of 
redundancy  dissolves  into  the  learning  “solution”  and,  with  the  learner’s  help,  the  solute  counterbalances  errors 
caused  by  acquisition,  processing,  and  retrieval  noise  in  the  selection  phase.  Here,  the  instructional  signal  provides 
redundant  message  cues  aimed  primarily  at  helping  the  learner  select  the  instructional  signal.  Similarly,  the  second 
layer  of  redundancy  in  an  instructional  message  can  reduce  the  effects  of  acquisition,  processing,  and  retrieval  noise 
in  the  analysis  phase  when  the  learner  exerts  information-processing  effort.  Now,  the  instructional  signal  includes 
message  cues  intended  to  help  the  learner  analyze  the  message.  In  the  final  phase,  the  third  layer  of  message 
redundancy,  in  conjunction  with  even  greater  information-processing  effort  on  behalf  of  the  learner,  neutralizes 
acquisition,  processing,  and  noise  at  the  synthesis  phase.  Here,  the  instructional  message  includes  cues  that  help  the 
learner  to  synthesize  the  content  of  a message. 
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By  the  time  the  message  has  reached  the  synthesis  phase,  all  redundant  layers  have  dissolved  and  the 
learning  “solution”  has  reached  its  saturation  point.  Meanwhile,  the  learner  ideally  has  become  a more  motivated 
participant  in  the  learning  (NE)  and  is  thinking  deeply  about  the  material.  This  state  of  engagement  thickens  the 
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Figure  8.  While  held  in  synthesis , 
core  begins  to  attract  solute 
from  saturated  solution. 
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Figure  9.  Schema  grows,  pulling 
together  surrounding  elements 
in  the  channel. 
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Figure  10.  A schema,  unique  to  the 
learner,  is  formed,  leaving  behind  some 
of  the  positive  affects  of  learning  (NE). 


saturated  learning  solution  and  slows  the  message  to  a stop.  These  conditions,  when  they  exist,  set  the  stage  for  the 
core  knowledge  to  seed  the  development  of  a crystallized  schema.  Figures  8,  9,  and  10  illustrate  how  the  message 
held  in  synthesis  — now  dissolved  down  to  its  core  knowledge — “crystallizes”  into  a schema  in  the  enriched 
solution.  During  synthesis,  learners  draw  upon  the  acquired  content,  processed  contexts,  and  retrieved  constructs 
that  are  the  products  of  the  chemical  reactions  at  each  learning  phase.  Learners  use  these  materials  to  build  and 
develop  their  own  structures,  or  “crystallized”  understandings  of  the  core  knowledge  conveyed  in  the  message.  Like 
crystals,  no  two  schemas  are  alike;  each  schema  formed  will  be  unique  to  the  learner.  Thus,  as  the  message  moves 
through  the  learning  phases,  it  is  processed  more  deeply  until  it  forms  a new  schema.  That  schema  is  then  passed  on 
to  long-term  storage. 

When  a message  with  very  little  redundancy  enters  the  system,  it  is 
likely  to  run  into  instructional  communication  problems  at  each  learning 
phase.  Messages  with  insufficient  selection-level  content,  context,  and 
construct  redundancy  do  not  evoke  enough  information-processing  effort 
from  the  learner  to  overcome  all  of  the  potential  for  noise.  Likewise,  these 
“chemical  reactions”  produce  very  little  positive  residual  (NE)  to  slow 
message  transit.  As  the  accelerating  message  reaches  the  analysis  phase,  the 
little  analysis-level  redundancy  it  contains  fails  to  neutralize  still  more  noise 
potential  and  doesn’t  slow  the  message  down  for  deeper  processing.  By  the 
time  the  message  speeds  through  synthesis,  its  lack  of  redundancy  has  left 
noise  potential  behind  in  each  learning  phase. 

While  it  is  possible  for  the  learner  to  defeat  some  of  the  noise 
potential  and  slow  the  message  for  deeper  processing  without  the  aid  of 
redundancy,  the  chemical  reaction  “formulas”  suggest  that  this  will  likely 
require  that  the  learner  supply  more  than  just  self-motivated  information- 
processing effort.  In  order  to  yield  sufficient  amounts  of  positive  residual  to 
slow  the  message,  the  learner  will  probably  also  have  “fill  in”  the  Cn,  Cx, 
and  Cs  parts  of  the  equation  with  dedicated  attention,  preexisting  strategies 
for  processing  the  new  information,  and  prior  understandings  of  the  topic. 

Further,  if  any  channel  noise  potential  is  not  overcome  during  processing,  it 
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Figure  11.  Even  if  the  core  remains 
in  the  channel  long  enough  to 
crystallize,  potential  noise 
remaining  in  the  channel  may  also 
act  as  ,f seeds,  ” increasing  the 
uncertainty  in  the  situation. 
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will  crystallize  along  with  the  core  during  synthesis,  depriving  the  message  crystal  of  needed  solute.  Ultimately,  the 
errant  structures  increase  the  uncertainty  in  the  situation,  making  it  difficult  for  the  learner  to  separate  the  deprived 
core  knowledge  structure  from  the  noise  (see  Figure  1 1). 

The  more  likely  scenario,  however,  is  that  the  weight  and  density  of  a message  with  very  little  redundancy 
will  cause  it  to  pass  through  the  system  unimpeded  and  largely  unprocessed.  If  a learner  is  not  interested  in,  curious 
about,  or  engaged  by  what  dissolves  from  the  message,  he  or  she  will  not  exert  the  information-processing  effort  that 
slows  it  down  at  each  learning  phase  for  deeper  processing  and  reflection.  While  what’s  left  of  the  message  core 
may  actually  end  up  somewhere  in  long-term  store,  its  mass  and  velocity  mean  that  — if  it  can  subsequently  be 
located — it  may  well  be  buried  too  far  out  of  reach  to  be  easily  retrieved. 

Like  message  schemas,  the  noise  that  crystallizes  in  the  channel  eventually  begins  to  make  its  way  into 
long-term  store,  leaving  behind  even  more  potential  for  noise  in  subsequent  learning.  When  noise  structures 
(misinterpretations,  misunderstandings)  find  their  way  into  long-term  store,  they  can  be  difficult  to  extract  later. 
Instructional  technologies  that  are  not  “front  loaded”  to  subdue  acquisition,  processing,  and  retrieval  noise  are 
unlikely  to  produce  outcomes  that  match  their  goals  (Dick  & Carey,  1990).  Effective  instructional  designers 
recognize  this  problem  and  build  into  the  message  the  redundancies  needed  to  resolve  those  problems  should  they 
occur.  Sound  may  play  a role  in  such  design.  This  model  may  help  to  illustrate  that  role  by  clarifying  the  operations 
and  limiting  factors  within  the  instructional  communication  system. 

A Cautionary  Note:  The  Function  of  a Model 

The  model  we  propose  is  intended  to  help  the  reader  conceptualize  the  role  of  redundancy  in  enhanced 
message  transmission.  Britt  (1997)  maintained  that  a model  — through  simplification,  explicitness,  and 
reformulation — provides  an  effective  way  to  sort  out  the  chaos  of  systems  that  are  too  complex  to  deal  with 
directly.  Because  explicit  systems  models  can  show  the  repeating  patterns  and  relationships  among  the  parts,  they 
can  help  one  understand  the  true  complexity  of  the  problem  or  situation.  Salisbury  (1996)  argued  that  to  be  useful, 
however,  a model  must  represent  all  of  the  system’s  components  and  the  relationships  between  them  simply  enough 
to  be  understandable.  The  model  must  reduce  complexity  and  ambiguity  sufficiently  so  as  to  make  analysis  and  the 
prediction  of  system  behavior  possible.  But  simplifying  real-world  complexity  poses  a dilemma.  If  a model  is  too 
simplistic,  the  relationship  of  the  model  to  its  real-world  counterpart  becomes  tenuous.  When  this  occurs, 
predictions  of  system  behavior  based  on  the  model  can  be  grossly  inaccurate.  Thus,  we  cannot  expect  to  model  a 
system  precisely.  Modeling  is  not  about  precision  but,  instead,  about  tentatively  determining  which  things  are 
important  to  consider  in  capturing  the  essence  of  the  system.  A model  cannot  provide  final  answers.  As  knowledge 
is  accumulated  and  relevant  areas  of  the  modeled  system  are  clarified,  the  model  is  almost  always  modified. 

We  suggest  that  this  model  may  prove  useful  in  deciding  how  to  incorporate  message  redundancy  through 
the  use  of  sound  cues  in  instructional  software.  Before  one  can  do  that,  however,  the  model’s  assumptions  should  be 
tested  against  what  is  known  about  “real  world”  instructional  software.  This  should  help  to  explicate  the  model  and 
reveal  more  specific  ways  sound  might  be  used  to  improve  the  effectiveness  and  efficiency  of  the  instructional 
communication  system. 
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Introduction 

Currently,  many  educators  suggest  that  learning  can  be  enhanced  if  the  learning  environment  includes  more 
interactive,  student-centered,  and  engaging  activities  where  learners  construct  their  understanding  rather  than  more 
traditional  methods  of  teacher-centered,  direct  instruction.  Inherent  is  a paradigm  shift  from  more  historical 
teaching  methods  to  an  environment  where  teachers  relinquish  control  and  learners  accept  responsibility  for 
learning.  Many  agree  that  this  promotes  more  ownership  and  stimulates  more  meaningful  learning.  However, 
engaging  in  such  a learning  environment  presents  challenges  for  both  the  teacher,  who  designs,  develops,  and 
facilitates  this  complex  environment,  and  learners  who  must  interact  and  take  responsibility  for  constructing  their 
understanding. 

This  paper  describes  a study  conducted  by  Northern  Arizona  University’s  Educational  Technology  faculty 
regarding  training  teachers  for  the  integration  of  technology  and  the  promotion  of  learner-centered  instruction. 
Participants  included  traditional  pre-service  students  enrolled  in  a required  “Technology  in  the  Classroom”  course 
and  veteran  teachers  engaged  in  professional  development  designed  to  provide  instruction  into  the  integration  of 
technology  into  the  classroom.  Instruction  modeled  the  integration  of  technology  from  a constructionist  perspective, 
and  provided  participants  the  opportunity  to  engage  in  activities  that  utilized  the  integration  of  technology.  The 
learning  environment  was  designed  to  provide  introduction  to  skills  and  practice  exercises  utilizing  computer 
applications  that  could  be  later  used  within  their  teaching  practice. 

Constructivist/Constructionist  Approach 

What  is  knowledge?  How  does  one  teach  this  knowledge  to  others?  Looking  at  educational  pedagogy 
from  a very  elementary  approach,  the  way  one  answers  the  first  question  will  determine  how  they  approach  the 
answer  to  the  second.  One  can  approach  the  answers  from  the  standpoint  that  knowledge  exists  outside  of  the 
learner,  there  are  fundamental  truths  and  teaching  is  helping  learners  master  them.  If  this  is  a person’s  view  of 
knowledge,  then  teaching  usually  takes  the  form  of  direct  instruction  and  the  goals  center  around  students  acquiring 
and  repeating  factual  information.  Most  printed  textbooks  are  designed  for,  and  many  teachers  are  trained  in,  this 
model.  Students  usually  read  or  are  told  factual  information,  and  then  repeat  this  information  as  a part  of 
assessment.  This  model  of  knowledge,  often  referred  to  as  the  objectivist  model,  works  well  when  the  objectives  to 
be  met  result  in  a type  of  informational  memorization. 

One  can  also  view  knowledge  as  something  beyond  a set  of  facts,  or  concepts,  or  laws  that  are  to  be 
memorized.  One  can  possess  a view  of  knowledge  that  incorporates  an  understanding  of  causes  and  effects 
involving  ideas  and  actions  that  requires  the  use  of  higher-order  or  critical  thinking  skills.  This  view  does  not 
conceive  knowledge  as  something  that  exists  independent  of  a knower.  Zahorik  (1995,  pp.  11-12)  summarized  this 
view  of  knowledge  in  the  following  way: 

“Knowledge  is  constructed  by  humans.  Knowledge  is  not  a set  of  facts,  concepts,  or  laws  waiting 
to  be  discovered.  It  is  not  something  that  exists  independent  of  a knower.  Humans  create  or 
construct  knowledge  as  they  attempt  to  bring  meaning  to  their  experience.  Everything  that  we 
know,  we  have  made. 

Knowledge  is  conjectural  and  fallible.  Since  knowledge  is  a construction  of  humans  and  humans 
are  constantly  undergoing  new  experiences,  knowledge  can  never  be  stable.  The  understandings 
that  we  invent  are  always  tentative  and  incomplete. 

Knowledge  grows  through  exposure.  Understanding  becomes  deeper  and  stronger  if  one 
tests  it  against  new  encounters.” 
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This  model  of  knowledge  is  often  referred  to  as  the  constructivist  model.  Constructivism’s  central  idea  is 
that  human  learning  is  constructed,  that  learners  build  new  knowledge  upon  the  foundation  of  previous  learning. 
The  constructivist  model  relies  on  cognitive  psychology  for  much  of  its  theoretical  foundations  and  has  roots  in 
philosophy,  sociology,  and  education.  It  is  important  to  understand  the  implications  this  view  of  learning  has  for 
teaching.  The  Southwest  Educational  Development  Laboratory  News  (SEDLetter)  in  August,  1996  stated: 

“First,  teaching  cannot  be  viewed  as  the  transmission  of  knowledge  from  enlightened  to 
unenlightened.... 

Second,  if  learning  is  based  on  prior  knowledge,  then  teachers  must  note  that  knowledge 
and  provide  learning  environments  that  exploit  inconsistencies  between  learners’  current 
understandings  and  the  new  experiences  before  them.... 

Third,  if  students  must  apply  their  current  understandings  in  new  situations  in  order  to 
build  new  knowledge,  then  teachers  must  engage  students  in  learning,  bringing  students’ 
current  understandings  to  the  forefront.  Teachers  can  ensure  that  learning  experiences 
incorporate  problems  that  are  important  to  students,  not  those  that  are  primarily  important 
to  teachers  and  the  educational  system.... 

Fourth,  if  new  knowledge  is  actively  built,  then  time  is  needed  to  build  it...” 

In  educational  pedagogy,  the  reality  of  the  situation  is  that,  teachers  find  themselves  in  both  the 
objectivist’s  camp  and  the  constructivist’s  camp  depending  upon  the  learning  objectives  of  the  moment.  There  are 
times  in  our  classrooms  that  our  objectives  are  such  that  we  are  actively  involved  in  the  “transmission  of  knowledge 
from  enlightened  to  unenlightened.”  There  are  others  times  that  our  learning  objectives  are  such  that  we  do  our  best 
to  create  situations  where  “students  must  apply  their  current  understandings  in  new  situations  in  order  to  build  new 
knowledge.”  Teaching  is  often  described  as  being  an  art.  The  art  of  becoming  a master  teacher  can  be  seen  as  an 
awareness  of  when  to  be  in  one  camp  or  the  other  and  an  understanding  of  how  to  be  effective  no  matter  what  camp 
one  is  in. 

In  contemporary  education,  there  has  been  added  to  this  complexity  of  teaching  and  learning  the  concept  of 
the  integration  of  modem  technology.  For  teachers  who  are  at  the  moment  in  the  objectivist’s  camp,  technology 
becomes  a tool  for  a more  effective  way  of  transmitting  knowledge.  In  this  context,  the  integration  of  technology 
usually  takes  the  form  of  some  type  of  PowerPoint®  presentation  or  the  use  of  some  other  multimedia  presentation 
software  to  supplement  teacher-centered  instruction.  But  for  those  times  when  a teacher  views  knowledge  from  a 
constructivist  perspective,  the  question  then  becomes,  how  can  technology  be  effectively  integrated? 

In  social  and  developmental  psychology,  according  to  von  Glasersfeld  (1994),  constructivist  models  view 
the  learner  as  a builder  of  knowledge,  not  a passive  receptor,  but  an  active  constructor.  Two  important  notions  orbit 
around  the  simple  idea  of  constructed  knowledge: 

“The  first  is  that  learners  construct  new  understandings  using  what  they  already  know. 

There  is  no  tabula  rasa  on  which  new  knowledge  is  etched.  Rather,  learners  come  to 
learning  situations  with  knowledge  gained  from  previous  experience,  and  that  prior 
knowledge  influences  what  new  or  modified  knowledge  they  will  construct  from  new 
learning  experiences.  The  second  notion  is  that  learning  is  active  rather  than  passive. 

Learners  confront  their  understanding  in  light  of  what  they  encounter  in  the  new  learning 
situation.  If  what  learners  encounter  is  inconsistent  with  their  current  understanding, 
their  understanding  can  change  to  accommodate  new  experience.  Learners  remain  active 
throughout  this  process:  they  apply  current  understandings,  note  relevant  elements  in  new 
learning  experiences,  judge  the  consistency  of  prior  and  emerging  knowledge,  and  based 
on  that  judgment,  they  can  modify  knowledge  (SEDLetter,  August,  1996).” 

If  learning  is  a constructive  process,  and  instruction  must  be  designed  to  provide  opportunities  for  such 
construction,  then  how  can  technology  be  integrated  into  the  instructional  processes  such  that  it  promotes  teachers  to 
teach  in  “constructivists  ways?”  The  answer  may  come  form  a series  of  research  studies  described  as 
constructionism. 

In  the  1960’s,  Seymour  Papert  and  colleagues  initiated  a research  project  on  how  children  think  and  learn 
and  to  develop  educational  approaches  and  technological  tools  to  help  those  children  learn.  From  this  beginning  has 
evolved  a theoretical  foundation,  which  has  become  known  as  constructionism.  The  term  constructionism,  first 
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coined  by  Papert  (1991),  involves  two  main  tenets.  First,  it  affirms  the  constructivists’  view  of  learning  and  asserts 
that  knowledge  is  not  simply  transmitted  from  teacher  to  student,  but  actively  constructed  by  the  mind  of  the  learner 
(Kafai  and  Resnick,  1996).  To  this  constructionism  adds  the  idea  that  people  construct  new  knowledge  with 
particular  effectiveness  when  they  are  engaged  in  constructing  personally  meaningful  products  (Bruckman  & 
Resnick,  1995).  Thus  constructionism  involves  two  intertwined  types  of  construction:  the  construction  of 
knowledge  in  the  context  of  building  personally  meaningful  products  (Kafai  and  Resnick,  1996).  It  is  through  this 
avenue  of  “constructing”  that  technology  can  be  integrated  into  the  instructional  processes  such  that  it  promotes 
teachers  to  teach  from  a constructivist  model. 

Participants 

The  integration  of  computer  technology  into  PreK-12  education  has  been  described  as  one  way  to  promote 
a learner-centered  environment  where  the  computer  acts  as  a tool  that  possesses  a cache'  of  knowledge  and  the 
teacher  introduces  and  moderates  ill-structured  problems  and  encourages  methods  for  learner  engagement. 
Described  as  both  constructivist  learning  theory  and  constructionist  methods,  interactive  learning  activities  within 
this  environment  include  developing  meaningful  products  through  student  publishing,  access  to  vast  resources, 
engaging  in  simulations,  and  utilizing  communication  systems  for  peer  collaboration. 

However,  computer  technology,  specifically  productivity  software  such  as  MS  Office,  Claris  Works,  and 
the  variety  of  authoring  tools  are  updated  generally  every  other  year.  In  addition,  various  computer  networks  are 
seldom  configured  the  same,  which  provides  a variety  of  pathways  and  location  names  for  file  management. 
Although  there  are  many  similarities  between  these  tools  and  learners  can  develop  crossover  skills,  developing 
instruction  based  upon  any  specific  tool  or  application  within  any  particular  system  seems  unwise.  Rather,  it  seems 
prudent  to  promote  the  learner's  understanding  of  the  concepts  that  are  the  foundation  of  the  applications  and  file 
management  systems.  Therefore,  it  has  been  the  goal  for  these  authors,  engaged  in  both  new  teacher  preparation  and 
professional  development  programs  for  in-service  teachers,  to  model  constructionist  principals  that  utilize  the 
integration  of  technology  in  a more  student-centered  learning  environment.  It  is  hoped  that  by  providing  this  type  of 
learning  environment,  the  learners  will  enhance  their  teaching  practice  by  realizing  they  can  use  these  tools  in  any 
environment,  regardless  of  platform,  application,  or  network  system. 

Participants  in  this  study  included  traditional  pre-service  students  enrolled  in  a required  "Technology  in  the 
Classroom"  course  and  veteran  teachers  engaged  in  professional  development  designed  to  provide  instruction  into 
the  integration  of  technology  into  the  classroom.  The  learning  environment  was  based  upon  constructionist 
principals  where  both  groups  of  learners  were  engaged  in  developing  meaningful  products  that  provided  an 
introduction  to  skills  and  practice  utilizing  computer  applications  that  could  be  later  integrated  within  their  teaching 
practice.  The  authors  also  considered  themselves  participants  in  this  study  as  their  instruction  modeled  the 
integration  of  technology  from  a constructionist  perspective,  and  provided  participants  the  opportunity  to  engage  in 
activities  that  utilized  the  integration  of  technology. 

Pre-service  Students 

All  elementary  education  majors  enroll  in  ETC  447,  Technology  in  the  Classroom,  as  a requirement  of  their 
program  of  studies.  It  is  a three-credit  hour  course  usually  taken  during  the  third  or  fourth  year  of  their  teacher 
education,  and  before  they  student  teach.  Classes  are  taught  in  a lab  of  22  Motorola  Macintosh  clones,  equipped  with 
printers,  scanners,  a Proxima  display  unit,  and  digital  cameras.  A network  that  includes  access  to  CD-ROM 
software,  student  and  instructor  folders,  and  fast  access  to  the  Internet  and  World  Wide  Web  links  these 
technologies.  Eight  to  eleven  sections  of  the  course  are  offered  each  semester  and  summer  sessions  to  accommodate 
traditional,  cohort,  and  alternative  partnership  elementary  teacher  education  programs. 

ETC  447  has  evolved  from  a skills-based  course,  in  which  students  individually  completed  activities  by 
following  a manual  of  scripted  instructions,  to  a classroom  simulation  of  a Multimedia  Content  Development 
Company,  where  student  teams  collaboratively  complete  content-centered  projects  using  carefully  integrated 
technology  tools.  It  is  our  vision  to  implement  an  educational  technology  course  that  models  authentic  practice 
through  hands-on  activities  and  social  interaction.  This  teaching  strategy  enables  our  students  to  "do"  technology 
and  "be"  desktop  publishers  and  multimedia  developers  and  database  managers  by  using  technology  tools  in  a 
supportive  learning  environment. 

In-service  Teachers 

The  veteran  teachers  in  this  study  were  a group  of  twenty-one  elementary  school  teachers  practicing  at  a 
school  district  in  southern  Arizona.  They  were  screened  and  accepted  into  the  eighteen-month  technology 
professional  development  project  with  four  university  educational  technology  faculty.  If  accepted  through  the 
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application  and  screening  process  each  participant  received  an  equipped  technology  cart  and  software  for  their 
classroom  in  return  for  their  long-term  commitment  to  hands-on  participation  in  technology  integration  activities 
and  training. 

To  build  cohesion,  enhance  teamwork,  and  stimulate  ownership  of  the  professional  development  project  a 
systems  approach  to  learning  (Senge,  1990)  was  taken.  The  Learning  Team,  as  they  later  called  themselves,  was 
given  time  during  each  visit  to  discuss  and  plan  future  topics  for  curriculum  based  upon  their  group  decided  needs. 
The  educational  technology  faculty  visited  on  site  four  times  during  each  of  the  first  two  semesters,  and  then  hosted 
the  in-service  teachers  for  a weeklong  summer  institute  on  campus.  Future  plans  include  four  more  visits  in  the 
2000  - 2001  academic  year  to  complete  the  professional  development. 

The  twenty-one  teachers  participated  in  strategies  similar  to  those  utilized  in  the  pre  service  ETC  447 
model  of  technology  integration.  In  self-selected  small  groups  they  “became”  travel  specialists  or  planners  of  a 
lecture  series  and  practiced  word  processing,  desktop  publishing,  database,  and  spreadsheet  skills  imbedded  in  the 
project.  The  teachers  chose  their  travel  destinations  to  investigate  and  promote,  or  lecture  series  notable  speakers,  so 
that  the  content  was  relevant  to  them.  The  final  product  was  a group  multimedia  presentation  of  their  project. 

Data  Collection 

This  paper  details  a collaborative  action  research  approach  to  (Oja  & Smulyan,  1989)  investigating 
teaching  technology  integration  from  a learner-centered  constructionist  perspective.  Both  quantitative  and 
qualitative  data  were  collected  through  self-report  instruments,  email  communications,  observations  and  anecdotal 
notes  of  instructional  sessions,  group  meetings  and  personal  interactions.  The  purpose  of  an  action  research 
approach  is  to  provide  a better  understanding  of  the  interactive  processes  and  promote  the  improvement  of 
conditions  for  the  participants  of  the  study.  Therefore,  data  were  collected  from  the  role  of  observer/participant. 
This  provided  the  opportunity  to  observe  and  help  support  the  authors  in  fostering  a better  understanding  and 
improve  the  learning  process  in  this  particular  study. 

Surveys,  observations,  email  communications,  anecdotal  notes,  and  personal  interactions  provided  both 
quantitative  and  qualitative  data  that  gave  a detailed  account  of  learning  experience  regarding  the  instructional 
methods  during  the  classes  and  professional  development  program.  The  qualitative  data  from  observations,  email 
communications,  anecdotal  notes,  and  personal  interactions  provided  detail  on  the  participants. 

Quantitative  data  collected  in  the  Center  for  Excellence  in  Technology  Survey  provided  basic  descriptions 
of  the  participating  learners  of  this  study.  The  Center  for  Excellence  in  Education  Technology  Survey  is  a self- 
report  that  provides  information  (5-point  likert  scale)  regarding  the  learners'  comfort  using  technology.  Information 
on  specific  technology  is  also  reported.  For  example,  participants  report  on  their  self-efficacy  using  basic  computer 
skills  (i.e.  word-processing,  email,  and  CD-ROMs). 

In  addition,  The  Stages  of  Concern  about  the  Innovation  Questionnaire  developed  by  Hall,  George  and 
Rutherford  (1977)  was  used  to  measure  the  professional  development  participating  teachers'  process  of  being 
selected  to  be  technology  innovators  for  their  district.  This  questionnaire  assesses  seven  hypothesized  stages  of 
concern  about  an  innovation  that  an  individual  moves  through  when  adopting  a process  or  product  innovation,  i.e., 
technology.  The  seven  stages  are:  (0)  Awareness,  (1)  Informational,  (2)  Personal,  (3)  Management,  (4) 
Consequence,  (5)  Collaboration,  and  (6)  Refocusing.  This  questionnaire  utilizes  an  eight-point  scale  of  0,  1 (not  true 
of  me  now),  2,  3,  and  4 (somewhat  true  of  me  now),  and  5,  6,  and  7 (very  true  of  me  now).  The  progression  from 
stage  to 'stage  indicates  the  participants’  ideas  that  go  from  unrelated  concerns  about  technology  usage  to  the  total 
involvement  with  technology  and  its  impact  on  the  learning  process. 

In  the  first  stage,  the  Awareness  stage,  little  concern  about  or  involvement  with  the  innovation  is  suggested. 
The  Informational  stage,  the  second  stage,  shows  a general  awareness  of  the  innovation  and  interest  in  learning  more 
detail  about  it.  In  the  third  stage,  the  Personal  one,  the  individual  is  uncertain  about  the  demands  of  the  innovation, 
his  or  her  role  with  the  innovation  and  his  or  her  adequacy  to  meet  the  innovation's  demands.  The  fourth  stage, 
Management,  focuses  the  attention  on  the  processes  and  tasks  of  using  the  innovation  and  the  best  use  of 
information  and  resources.  In  the  fifth  stage,  Consequence,  the  focus  is  on  the  impact  that  the  innovation  may  have 
on  the  students’  outcomes.  Collaboration,  the  sixth  stage,  focuses  on  coordination  and  cooperation  with  others 
regarding  the  use  of  the  innovation.  Finally  in  the  Refocusing  stage,  the  seventh  stage,  the  focus  is  on  exploration  of 
more  universal  benefits  from  the  innovation,  including  the  use  of  alternative  ideas  to  the  proposed  or  existing  form 
of  innovation. 
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Discussion 

There  were  many  similarities  between  both  the  campus-based,  pre -service  group  and  the  professional 
development  group  in  regard  to  their  self-efficacy  of  using  computers.  For  example,  there  was  no  significant 
difference  between  the  pre-service  and  the  in-service  teachers  in  their  self-efficacy  reports  of  computer  use.  As 
table  1 indicates,  both  groups  demonstrated  a high  level  of  self-efficacy  for  using  computers  for  general  tasks  - 
4.201176471  and  4.1 1047619  out  of  5. 


Pre-service  Teachers  n=34 

In-Service  Teachers  n=22 

Mean 

4.201176471 

4.11047619 

Standard  Deviation 

0.532528359 

0.67015279 

Table  1.  Computer  Use  Self-Efficacy 

Participants  in  both  groups  indicated  that  they  desired  less  constructionist  methods.  Interestingly,  participants,  at 
times,  resisted  the  more  open  discovery  learning  methodology  and  indicated  on  several  occasions  that  they  would 
rather  be  provided  with  very  specific  "how  to”  skill  instruction  using  the  computers  and  software  applications. 

The  Professional  Development  Group 

Several  participants  indicated  that  they  wanted  to  know  how  to  use  the  software,  become  "experts"  or 
"gurus"  before  they  learned  how  to  integrate  the  technology  into  their  teaching  practice.  They  suggested  that 
learning  to  use  the  computer  and  software  was  an  independent  learning  objective  different  from  learning  how  to 
integrate  the  technology  into  the  classroom.  As  one  professional  development  participant  commented:  "I  need  to 
understand  how  to  use  this  equipment  before  I can  learn  how  to  integrate  it  into  my  teaching."  Indeed,  the 
professional  development  group  on  several  occasions  requested  listed  steps  for  using  specific  computer  applications. 
This  is  particularly  surprising  as  members  of  the  professional  development  group,  a)  were  chosen  for  the  program 
based  on  their  self-reported  technology  skills,  b)  demonstrated  generally  high  self-efficacy  for  computer  use  skills, 
and  c)  demonstrated  a Personal  and  Management  stage  at  the  80th  percentile  on  The  Stages  of  Concern  about  the 
Innovation  Questionnaire,  suggesting  that  they  have  high  technology-knowledge  of  innovating  people  who  had 
already  mastered  technology. 

The  participating  teachers  were  engaged  in  additional  technology  training  sessions  at  the  same  time 
conducted  by  various  other  providers  as  well.  Several  individuals  indicated  that  they  were  receiving  step-by-step 
handouts  and  worksheets  in  their  other  training  sessions  and  asked  why  they  weren't  being  provided  those  during 
this  professional  development  as  well. 

Indeed,  their  demand  to  learn  more  computer  skills  continued  to  the  point  that  their  district  technology 
program  director  made  available  online  skills  training  for  the  entire  professional  development  group  between  the  4th 
professional  development  session  and  the  week  long  summer  session,  as  indicated  by  this  email. 

"I  have  20  TEAM  members  signed  up  for  Teacher  Universe.  The  pre  training 
session  is  set  for  Saturday,  May  6,  2000  from  8:30  a.m.  to  noon.  I will  open  up  the 
training  to  5 more  people  today.  If  you  want  to  view  the  site  before  Saturday,  go  to 
http .7/ www. teachenmiverse.com  See  you  all  Saturday!" 

Their  desire  to  have  more  application  skill  training  was  constant  through  out  the  professional  development 
training  sessions.  During  a discussion  on  the  project  bulletin  board  about  their  taking  on  a specific  area  to  become  an 
expert,  as  this  teacher  stated: 

"I  also  think  some  of  our  [professional  development  group]  members  are  feeling 
a little  computer-deficient,  or  at  least  they  lack  confidence  in  their  own  skills.  Maybe  if 
we  had  had  some  application-based  training  going  on  concurrent  to  our  class,  it  would 
have  been  less  difficult." 

Another  teacher  may  have  hit  the  nail  on  the  head  by  responding: 

"I  think  one  of  the  reasons  people  are  uncomfortable  choosing  an  area  of 
"expertise"  is  the  term  itself.  I consider  myself  to  be  a damned  fine  teacher,  but  not  an 
"expert."  An  expert  in  my  mind  is  someone  who  knows  everything  about  that  topic  - 
someone  to  whom  I can  go  to  for  the  nitty  gritty.  I am  someone,  though,  that  you  could 
come  to  for  advice  on  teaching.  Maybe  we  need  to  consider  a different  term  for  'expert'." 

Interestingly,  this  suggests  that  in  an  area  such  as  technology,  which  changes  constantly,  individuals  may 
have  a difficult  time  seeing  themselves  as  ever  being  experts  or  having  enough  skills.  Certainly  the  participating 
teachers  of  the  professional  development  group  demonstrated  this.  One  of  the  NAU  faculty  members  specifically 
tried  to  address  this  at  one  of  the  professional  development  group  planning  sessions  with  the  teachers  when  he 
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stated;  "At  what  point  do  you  consider  yourself  an  expert?  I don't  have  all  of  the  knowledge.  I'm  constantly 
learning  new  technology  skills.  I don't  have  all  of  the  answers;  I just  have  more  than  you  at  this  point.  You  need  to 
realize  that  you  also  have  more  knowledge  and  skills  than  others,  and  to  them  you  are  a guru."  Indeed,  the  topic  of 
"How  do  we  get  them  to  realize  that  this  isn't  about  learning  skills,  but  rather  a shift  in  the  learning  paradigm?"  was 
discussed  often  during  project  planning  meetings  by  the  NAU  faculty. 

Participants  indicated  on  several  occasions  that  they  desired  specific  skill  set  instructions  because  they 
believed  it  would  be  easier  to  learn  a step-by-step  process  rather  than  having  the  skills  embedded  into  a more 
project-based  method.  Most  interestingly  however,  during  discussions  of  how  they  might  integrate  technology  into 
their  classrooms,  the  stated  that  they  thought  using  the  learner-centered  environment  (Travel  Agency  or  Lecture 
Series  Experts)  that  utilized  imbedding  the  computer  skills  into  a constructionist  model  utilized  by  the  NAU  faculty 
would  be  an  excellent  way  to  teach  their  students.  This  was  in  direct  contrast  to  how  they  demonstrated,  in  many 
ways,  that  they  preferred  to  be  taught  technology  instruction. 

The  University  Pre-service  Group 

The  purpose  of  providing  more  learner-centered  instruction  is  to  promote  greater  understanding  of  the  skills 
and  concepts.  Students  in  the  university  technology  course  taught  by  the  authors  are  provided  with  instruction  based 
upon  this  constructionist  model.  Similarly  to  the  professional  development  group,  this  group  indicated  their  self- 
efficacy  in  basic  computer  use  (m=4.201 176471  out  of  5,  sd=0.532528359),  but  also  demonstrated  the  desire  to 
learn  with  greater  emphasis  on  step-by-step  methodology.  The  end  of  the  semester  student  evaluations  included 
comments,  such  as  this  student's: 

"Less  content  - more  explanation.  More  help  sessions,  possible  written  step-by- 
step  instructions  for  programs  and  a frequently  asked  questions  handout.  More  visual 
aids,  such  as  handouts." 

Another  student  commented  that  he  would  have  preferred  the  methodology  utilized  by  the  adjunct  faculty 
and  stated: 

I observed  several  other  ETC  447  classes  while  working  on  class  projects  and 
noticed  a big  difference  between  my  class  and  the  others.  Our  class  spent  more  time 
utilizing  trial  and  error  to  learn  material.  Other  classes  spent  time  moving  through 
material  step  by  step.  Our  class  could  have  benefited  from  a little  more  step  by  step 
attention  to  limit  confusion." 

These  comments  suggest  that  many  students  desired  direct  instruction.  One  student  even  suggested  that  he 
wanted  more  lecture  time  by  stating;  "Dr.  T.  is  cool,  but  there  may  have  been  a little  bit  too  much  workload  for  the 
amount  of  lecture  time  used."  Another  student  seemed  to  miss  the  point  of  "constructionism"  entirely.  "I  would  like 
to  have  this  course  be  more  classroom  oriented.  I will  never  build  a computer  program  [lecture  series  project]  again 
in  my  life.  I can't  think  that  this  course  is  relevant  to  future  teachers." 

Not  preferring  or  not  understanding  the  learning  methodology  that  was  employed  and  modeled  in  this 
learning  environment  may  have  been  frustrating,  as  this  student  stated:  "This  course  forced  me  to  learn  a lot  about 
computers.  I had  to  figure  out  my  programs  by  myself  and  this  caused  much  frustration." 

Furthermore,  cooperative  leaning  activities  are  built  into  the  course  through  team  projects  and  the  faculty 
encourages  peer  support  and  collaboration.  However,  this  was  seen  as  not  getting  the  information  from  the 
instructor.  Indeed,  this  student  commented: 

"It  was  difficult  to  get  assistance  from  either  the  teacher  or  the  GA.  Most  of  the 
teaching  was  done  by  the  few  students  who  actually  understood  what  and  how  to  run  the 
programs,  which  they  learned  on  their  own  or  in  other  classes." 

This  student  commented  that:  "Some  things  taught  in  the  1st  part  of  the  semester,  I forgot  later.  It  would  be 
helpful  if  a handout  of  some  type  was  given." 

Even  though  these  comments  indicate  that  many  students  would  have  preferred  more  direct  instruction  and 
that  they  were  frustrated,  they  also  indicated  that  they  learned  a great  deal.  This  student  stated:  "I  probably  have 
learned  more  about  computers  and  technology  in  3 months  of  this  class  than  I have  in  21  years  of  life."  Another 
stated:  "The  course  is  a little  overwhelming  sometimes,  but  I really  learned  a lot."  Interestingly,  this  student  stated: 
"This  was  a great  course  that  really  helped  me  further  develop  my  computer  skills." 
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Conclusions 

Conclusions  drawn  from  this  study  suggest  that  currently  learners  may  not  have  enough  experience 
learning  with  the  integration  of  technology  to  feel  comfortable  to  take  responsibility  for  this  type  of  learner-centered 
environment.  The  participants  of  this  study,  especially  in  the  professional  development  group,  may  never  have  been 
taught  from  a constructionist  paradigm.  This  could  lengthen  the  time  it  takes  for  these  learners  to  feel  comfortable 
with  having  their  instructor  in  the  role  of  a facilitator  of  learning,  rather  than  the  giver  of  knowledge.  For  example, 
the  professional  development  group  presented  a challenge  for  the  NAU  faculty  not  because  of  the  lack  of  technology 
skills  among  some  participants,  but  because  of  their  lack  of  self-confidence  of  being  an  expert . Furthermore,  the 
university  pre-service  students  indicated  their  desire  for  more  direct  instruction  and  even  commented  on  their 
frustration  with  having  to  "figure  out  the  programs."  However,  for  those  students  who  saw  themselves  as  experts, 
the  learning  didn’t  appear  to  be  as  frustrating.  For  example,  the  student  who  thought  that  the  course  was  “great”  and 
that  it  helped  her  “ further  develop ” her  computer  skills.  This  indicates  that  she  believed  she  already  had  some 
computer  skills.  Perhaps,  her  awareness  of  her  self-efficacy  gave  her  the  self-confidence  to  explore  and  better 
supported  her  engagement  in  this  type  of  learning  environment. 

This  brings  up  new  questions  regarding  the  maturity  or  level  of  background  knowledge  needed  to 
understand  the  concept  of  the  integration  of  technology  within  the  PreK-12  classroom.  What  does  this  say  about 
how  we  prepare  our  future  teachers  or  provide  professional  development  for  our  in-service  teachers,  specifically  in 
the  complex  area  of  computer  technology?  Perhaps  as  the  professional  development  group  member  who  stated, 
"Maybe  we  need  to  consider  a different  term  for  "expert,"  we  educators  need  to  help  our  pre-service  and  in-service 
teachers  re-define  expertise  and  try  to  instill  greater  self-confidence  in  their  ability  to  learn  the  integration  of 
technology  into  instruction.  Although  these  learners  may  have  not  felt  comfortable,  they  did  indicate  that  they 
learned.  Indeed,  the  professional  development  group  even  indicated  that  they  would  probably  integrate  technology 
into  their  teaching  practice  in  a similar  fashion.  Because  of  the  rapid  development  of  computer  technology,  expert 
knowledge  is  constantly  changing.  Therefore,  learners  must  continually  learn  by  using  their  current  skills  and 
knowledge  to  explore  new  programs  and  technologies.  Educators  should  focus  on  helping  learners  build  upon  their 
self-efficacy  to  support  their  inquiry  within  this  new  paradigm. 
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Introduction 

Research  suggests  that  cooperative  learning  influences  student  achievement,  attitude,  and  perception  of 
social  support  and  self  esteem  (Johnson  and  Johnson,  1996).  The  success  of  cooperative  learning  has  been  well- 
documented  in  studies  with  K-12  students,  and  to  a lesser  extent,  with  college  students.  However,  there  has  been 
little  focus  on  cooperative  learning  in  the  population  of  adult  learners. 

Adult  college  re-entry  students  may  have  an  educational  experience  that  is  similar  to  traditional  age  college 
students  since  they  are  subject  to  the  same  course  content  and  academic  expectations.  Springer,  Stanne,  and 
Donovan’s  (1999)  meta-analysis  of  39  studies  of  cooperative  learning  in  undergraduate  science,  math,  engineering 
and  technology  settings  points  to  significant  positive  effects  of  small  group  learning  on  achievement,  persistence, 
and  attitude.  Wedman,  Hughes,  and  Robinson  (1993)  compared  systematic  cooperative  learning  to  direct  instruction 
with  undergraduate  education  majors  and  found  significant  differences  on  learning  outcome  scores.  Additionally, 
participants  in  this  systematic  cooperative  learning  setting  rated  working  with  a group  as  more  beneficial  to  their 
learning  than  listening  in  class.  Likewise,  McDonald,  Dansereau,  and  Spurlin’s  1985  study  of  undergraduate 
psychology  students  revealed  significant  differences  favoring  a systematic  cooperative  strategy.  The  favorable 
difference  in  achievement  was  true  for  measures  of  both  initial  learning  and  for  transfer  to  a subsequent  individual 
learning  situation. 

On  the  other  hand,  Klein  and  Schnackenberg  (2000)  found  that  undergraduates  working  alone  scored 
significantly  higher  than  those  working  in  cooperative  dyads,  when  measured  on  knowledge  items,  but  no 
differences  on  application  items.  Klein  and  Pridemore  (1992)  found  no  significant  main  effect  in  achievement  for 
undergraduates  working  in  groups  or  individually,  but  a significant  interaction  according  to  differences  in  need  for 
affiliation  on  certain  achievement  measures.  Klein,  Erchul,  and  Pridemore  (1994)  investigated  cooperative  learning 
and  type  of  reward  with  undergraduate  education  majors  and  found  that  students  who  worked  alone  performed  better 
on  achievement  measures,  with  significant  differences  according  to  type  of  reward:  task  reward,  performance 
reward,  or  no  reward. 

Adult  re-entry  students  may  have  learning  needs  that  are  quite  different  from  those  of  traditional  college 
students.  Brookfield  (1986)  and  Knowles  (1980)  suggest  that  adult  learners  come  into  an  education  activity  with  an 
orientation  different  than  younger,  traditional  students.  They  posit  that  adults  are  motivated  to  learn  in  situations 
where  they  can  relate  what  they  know  from  life  experience  to  new  information,  and  similarly  use  new  information  in 
service  of  real-life  situations.  Cooperative  learning  may  provide  an  arena  in  which  life  experience  can  be  shared 
among  adult  learners.  Research  conducted  in  specific  populations  of  adult  learners,  therefore,  should  be  considered 
along  with  research  on  college-level  learning. 

Thompson  and  Scheckley  (1997)  investigated  preference  for  cooperative  learning  among  baccalaureate 
(BSN)  students,  according  to  their  status  as  traditional  college  students,  adults  with  no  prior  nursing  education,  and 
adults  with  prior  nursing  education.  For  the  most  part,  the  preference  for  classroom  teaching  methods  did  not  vary 
significantly  between  traditional  college  students  and  adults  with  no  prior  nursing  education.  However,  the  adults 
with  prior  nursing  education  rated  their  classroom  experience  with  cooperative  learning  significantly  higher  than  did 
either  the  traditional  students  or  adults  with  no  prior  nursing  education. 

Cole  and  Smith  (1993)  found  no  significant  differences  in  the  achievement  of  adult  students  in  business 
English  classes,  but  did  observe  increased  positive  interactions,  compared  to  the  control  group  who  participated  in 
an  interactive  classroom  where  the  teacher  called  on  students  at  random.  Cavalier,  Klein  and  Cavalier  (1995)  found 
results  in  favor  of  cooperative  learning  for  adults  in  a technical  training  setting.  Trainees  who  used  structured 
cooperative  groups  during  technical  training  performed  better  on  a posttest  than  did  those  who  worked  in 
unstructured  cooperative  groups.  Additionally,  there  were  significant  differences  favoring  structured  groups  on  four 
of  nine  attitude  measures,  structure  groups  exhibited  increased  social  and  cognitive  behaviors. 
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The  ability  to  learn  while  working  in  groups,  and  to  be  motivated  by  the  experience  of  cooperative  learning 
settings,  may  be  influenced  by  status  variables  such  as  need  for  affiliation  or  sociability  (McClelland,  1976;  Jackson, 
1974).  A reasonable  expectation  is  that  high  affiliation  students  prefer  group  work  more  than  low  affiliation 
students.  Sutter  and  Reid  (1969)  reported  that  college  student  with  high  levels  of  sociability  performed  better  using 
cooperative  learning  with  computer  assisted  instruction.  Need  for  affiliation  as  a status  variable  during  cooperative 
learning  has  been  investigated  with  mixed  results.  Predictably,  Klein  and  Schnackenberg  (2000)  found  that  low 
affiliation  undergraduates  reported  higher  continuing  motivation  for  working  alone,  and  high  affiliation  students 
reported  greater  continuing  motivation  for  working  with  another  person.  Klein  and  Pridemore  (1992)  found  high 
affiliation  students  working  alone  scored  significantly  lower  on  application  items  compared  to  high  affiliation 
students  working  individually,  and  low  affiliation  students  in  both  group  and  individual  conditions.  Also,  students 
who  worked  cooperatively,  regardless  of  affiliation  motive,  reported  significantly  greater  overall  satisfaction  than 
those  who  worked  individually.  Nakanishi  (1988)  studied  undergraduate  speech  communications  students  working 
in  cooperative  learning  groups  that  were  experimentally  manipulated  to  represent  varying  conditions  of  motivation. 
On  measures  of  task-persistence,  there  were  significant  differences  in  favor  of  students  who  were  prompted  that  the 
task  was  valuable  and  that  a certain  level  of  effort  expenditure  would  lead  to  a certain  level  of  performance. 

The  opportunity  to  work  in  small  informal  groups  provides  an  avenue  to  interact  with  peers,  fulfilling  some 
of  the  needs  of  adult  learners,  and  in  particular,  the  needs  of  students  with  high  affiliation  motive.  When  observed 
for  interaction  behaviors  within  the  cooperative  treatment,  Klein  and  Schnackenberg  (2000)  found  that  high 
affiliation  dyads  exhibited  significantly  more  on-task  behaviors,  and  also  exhibited  significantly  more  off-task 
behaviors  compared  to  low  affiliation  dyads.  Klein  and  Pridemore  (1992)  found  that  undergraduates  working 
cooperatively  spent  more  time  on  task  than  did  those  who  worked  individually,  without  regard  to  high  or  low  need 
for  affiliation.  Cavalier  et  al.  (1995)  found  significant  differences  in  social  and  cognitive  behaviors,  with  increased 
activity  among  the  structured  cooperative  groups  compared  to  unstructured  cooperative  groups.  Cole  and  Smith 
(1993)  observed  increased  camaraderie  among  students  working  in  informal  cooperative  groups,  noted  in  terms  of 
willingness  to  help  group  members,  expressing  excitement  over  improvement  in  one  another’s  test  scores,  and 
willingness  to  express  that  one  did  not  understand  a part  of  the  content. 

The  purpose  of  the  current  study  was  to  investigate  the  effect  of  informal  cooperative  learning  and  the 
affiliation  motive  on  achievement,  attitude  and  student  interactions.  The  study  attempts  to  extend  previous  research 
(Klein  and  Schnackenberg,  2000  and  Klein  and  Pridemore,  1992)  by  examining  the  relationship  between  affiliation 
motive  and  interactions  during  informal  cooperative  learning,  for  the  specific  target  audience  of  adult  re-entry 
students. 

Methods 

Participants 

In  this  study,  participants  were  109  undergraduate  business  majors  enrolled  at  a private  degree  completion 
university  for  adult  learners.  University  standards  indicate  that  these  students  are  a minimum  of  age  23,  have  been 
employed  for  two  years,  and  have  earned  a minimum  of  30  college  credit  hours  upon  admission. 

Procedures 

Based  on  the  affiliation  scale  of  the  Personality  Research  Form-E  (Jackson,  1974),  participants  were 
identified  as  having  a high  or  low  affiliation  motive.  After  blocking  by  high  and  low  affiliation  motive,  participants 
were  randomly  assigned  to  one  of  two  treatments — small  group  or  individual  practice.  In  the  small  group  treatment, 
students  worked  in  homogeneous  triads  (three  with  high  affiliation  motive  or  three  with  low  affiliation  motive)  with 
shared  materials,  and  in  the  individual  treatment  students  worked  individually  with  their  own  materials. 

The  experimental  period  consisted  of  two  20-minute  lectures  and  two  40-minute  practice  sessions  within 
the  treatment  conditions.  The  first  lecture  on  the  topic  of  organizational  structure  was  conducted  for  the  entire  class, 
then  participants  moved  to  their  small  group  or  individual  treatment  conditions  for  a 40  minute  practice  session. 
After  a break,  all  students  returned  to  their  regular  classroom  setting  to  receive  a 20-minute  lecture  on  the  topic  of 
employee  motivation.  Participants  then  moved  to  their  same  treatment  conditions  for  a 40-minute  practice  session 
on  this  second  topic.  During  both  practice  sessions,  student  interactions  were  observed  and  recorded.  Upon 
completion  of  the  second  practice  session,  all  participants  returned  to  their  regular  classroom  setting  and  were  given 
a posttest  and  attitude  survey.  Participants  in  both  groups  were  tested  together  in  the  same  room  and  performed 
individually  on  the  posttest,  without  the  use  of  the  workbooks,  notes,  or  textbook  reference. 
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Results 

Analysis  of  achievement  scores  among  the  four  treatment  conditions  indicated  no  significant  differences. 
The  equivalency  of  achievement  levels  is  likely  due  to  the  use  of  instructionally  designed  materials  which  included 
learning  objectives,  a timed  presentation  sequence  for  the  instructor’s  lecture,  a set  of  overhead  transparencies  to 
display  content  points,  and  student  workbooks  containing  practice  items  aligned  to  the  posttest. 

There  were,  however,  significant  differences  in  attitude.  Assessment  of  attitudes  toward  small  group 
practice  and  individual  practice  included  items  regarding  confidence,  continuing  motivation,  in-class  activities,  and 
homework  assignments,  with  six  of  the  eight  attitude  items  indicating  significant  differences.  Participants  in  the 
small  group  condition  reported  greater  confidence  than  participants  in  the  individual  condition,  p < .05,  and  greater 
continuing  motivation,  p < .01.  Results  also  revealed  a significant  interaction  between  practice  condition  and 
affiliation  motive  on  three  of  the  eight  attitude  items  regarding  enjoyment  of  the  practice  activity,  p < .02,  belief 
about  one’s  own  ability  to  learn,  p < .05,  and  preference  to  work  in  groups  during  classroom  activities,  p < .02.  For 
each  of  these  three  attitude  items,  follow-up  analyses  indicated  high  affiliation  participants  who  worked  in  a small 
group  had  more  favorable  attitudes  compared  to  high  affiliation  participants  working  individually,  as  well  as 
compared  to  low  affiliation  participants  working  individually.  Additionally,  there  was  a statistically  significant 
interaction  on  the  attitude  item  regarding  preference  to  work  in  groups  for  homework  assignments,  p < .05,  with 
high  affiliation  participants  preferring  groups  and  low  affiliation  participants  preferring  individual  work.  Follow-up 
analyses  indicated  no  significant  differences  among  the  treatment  condition  means  on  this  item. 

Regarding  student  interactions,  high  affiliation  triads  exhibited  significantly  more  on-task  behaviors 
(statements  intended  to  accomplish  the  task,  sharing  materials,  taking  turns)  than  did  low  affiliation  triads,  p < .01. 
However,  there  were  no  significant  differences  in  helping  and  off-task  behaviors.  The  qualitative  records  of  trained 
observers  revealed  that  participants  in  the  small  group  treatment  condition  were  physically  and  verbally  active 
during  their  timed  practice,  with  behaviors  such  as  “moved  to  the  other  side  of  the  table”  and  “held  up  the 
organizational  chart  to  demonstrate.”  Qualitative  records  for  participants  in  the  individual  practice  treatment  also 
indicated  some  physical  and  verbal  activity  during  their  timed  practice,  with  behaviors  such  as  “commiserated  about 
the  assignment”  and  “read  the  newspaper  when  finished  with  the  workbook.” 

Discussion 

The  purpose  of  this  study  was  to  investigate  the  use  of  small  group  practice  and  affiliation  motive  on  the 
achievement  and  attitude  of  adult  re-entry  students.  Regarding  student  achievement  on  selected  and  constructed 
response  items,  no  significant  differences  were  found.  Similarly,  Cole  and  Smith  (1993)  found  no  significant 
differences  in  group  v.  individual  work,  while  others  found  results  in  favor  of  working  alone  (Klein  and 
Schnackenberg,  2000;  Klein  and  Pridemore,  1992;  and  Klein  et  al.  (1994).  However,  the  benefits  of  formal  and 
informal  cooperative  learning  may  not  be  limited  to  effects  on  achievement. 

Attitudinal  factors  such  as  continuing  motivation,  confidence  and  satisfaction  may  well  have  an  impact  on 
academic  success  apart  from  achievement  scores.  Persistence  to  complete  a course  and  maintain  continuous 
enrollment  is  vital  to  adult  student  re-entering  college  after  an  absence  of  a few  or  several  years.  Results  on  five  of 
the  nine  attitude  measures  in  this  study  indicated  that  participants  favored  working  in  small  groups.  These  findings 
are  consistent  with  results  of  other  researchers  in  undergraduate  and  adult  learning  settings  (Klein  and  Pridemore 
,1992;  Cavalier  et  al.,  1995;  Thompson  and  Scheckley,  1997;  and  Klein  and  Schnackenberg,  2000).  The  results  of 
one  attitude  measure,  regarding  homework  assignments  between  classes,  revealed  an  interaction  between  affiliation 
motive  and  preference  for  small  group  or  individual  work.  As  anticipated,  high  affiliation  participants  preferred 
group  work  and  low  affiliation  participants  preferred  individual  work.  Klein  and  Schnackenberg  (2000)  found  a 
similar  interaction  between  affiliation  and  preference  for  group  or  individual  work. 

Several  researchers  have  noted  increased  positive  interaction  behaviors  during  cooperative  learning. 
Among  the  adult  learners  working  in  groups  in  this  study,  those  with  high  affiliation  motive  were  noted  to  exhibit 
significantly  more  on-task  interaction  behaviors  than  those  with  low  affiliation  motive.  However,  there  was  no 
difference  in  off-task  behaviors,  as  was  the  case  in  Klein  and  Schnackenberg’s  (2000)  study  with  undergraduates. 
Given  the  opportunity  to  participate  in  informal  small  groups,  adult  re-entry  students  may  use  the  task  as  a means  of 
affiliating. 

Implications  of  this  study  point  to  the  advisability  of  including  cooperative  learning  activities  in  higher 
education  for  adult  re-entry  students.  While  individual  achievement  was  not  influenced  by  practice  in  either  a small 
group  or  individually,  continuing  motivation  and  confidence  were  noted  to  be  greater  with  a group  experience. 
These  attitudinal  gains  may  positively  support  adult  learners  throughout  the  academic  and  social  experience  of 
completing  an  undergraduate  degree. 
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Abstract 

This  paper  describes  the  application  of  problem-based  learning  (PBL)  design  principles  and  the  inclusion 
of  teacher  and  student  scaffolds  to  the  design  and  implementation  of  a hypermedia-based  learning  unit  for  the  upper 
elementary  / middle  school  grades.  Results  suggest  that  PBL  may  be  an  effective  instructional  strategy  for  gifted 
and  talented  sixth  grade  students.  Student  scaffolding  did  not  seem  to  impact  achievement  or  student  attitudes , 
however,  teacher  scaffolding  appeared  to  increase  teacher  effectiveness,  confidence  and  attitudes. 

Introduction 

Changing  perspectives  in  curriculum  and  instruction  over  the  last  decade  have  focused  increasingly  on  the 
need  to  revitalize  K-12  instruction  through  a greater  focus  on  understanding  of  concepts  within  a real-world  context 
(Gallagher,  Sher,  Stepien  & Workman,  1995).  One  means  suggested  for  achieving  this  is  problem-based  learning 
(PBL).  PBL  advocates  maintain  that  students  engaged  in  such  learning  environments  develop  more  positive  attitudes 
toward  learning  (Sobral,  1995;  Kaufman  & Mann,  1997),  tend  to  focus  on  meaning  rather  than  recall,  and  achieve 
essentially  the  same  knowledge  level  as  those  receiving  traditional  instruction  (Gallagher  & Stepien,  1996).  These 
findings  suggest  that  PBL  environments  could  contribute  to  improving  student  learning  in  K-l  2 settings. 

Implementation  of  problem-based  learning  is  not  without  drawbacks,  however,  one  of  which  is  the 
difficulty  associated  with  providing  a sufficiently  rich  informational  environment  to  support  problem-based  inquiry 
(Hoffman  & Ritchie,  1997).  Hoffman  & Ritchie  (1997)  suggest  that  the  expanding  capacity  of  multimedia  may 
represent  one  means  for  providing  such  a robust  environment.  Multimedia  may  afford  additional  benefits  in  PBL 
environments  according  to  Spiro,  Feltovisch,  Jacobson,  and  Coulson  (1992).  They  suggest  that  the  multiple 
exposures  to  content  multimedia  provides  may  increase  student  learning  in  accordance  with  cognitive  flexibility 
theory.  Cognitive  flexibility  theory  proposes  that  in  open-ended  learning  environments,  individuals  may  become 
overwhelmed  initially  by  the  scope  of  available  information.  As  the  learner’s  understanding  of  the  problem  and 
available  resources  develops,  each  additional  exposure  supports  deeper  understanding. 

A second  drawback  associated  with  the  implementation  of  PBL  units  centers  on  the  lack  of  experience 
students  and  teachers  have  with  learning  within  open-ended  learning  environments.  For  students,  such 
unfamiliarity  may  result  in  confusion  and  uncertainty  as  to  how  to  proceed  (Sobral,  1995;  Hoffman  & Ritchie, 
1997),  while  for  teachers,  unfamiliarity  with  the  facilitator  or  knowledge  resource  role  PBL  methodologies  require 
may  pose  an  equally  difficult  challenge  (Hannafin,  Hall,  Land  & Hill,  1994).  Hannafin,  Land  and  Oliver  (1999) 
suggest  scaffolds,  supports  for  learning  efforts  when  engaging  in  open-ended  learning  environments,  may  enhance 
learner  effectiveness.  These  scaffolds  may  take  the  form  of  tools,  strategies  and  guides  as  means  for  addressing 
problems  associated  with  these  unfamiliar  student  and  teacher  roles. 

The  purpose  of  this  study  was  to  examine  the  following  research  questions: 

1 . Does  hypermedia-based  PBL  represent  an  effective  instructional  strategy  for  upper  elementary  / middle  school 
students? 

2.  What  are  student  and  teacher  attitudes  toward  the  PBL  instructional  unit? 

3.  How  are  student  and  teacher  scaffolds  utilized  during  implementation  of  the  PBL  unit? 
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Method 

Design 

Design  of  the  hypermedia-based  PBL  unit  was  based  on  two  models.  The  first  was  the  design  model  for 
creating  problem-based  learning  units  developed  by  The  Center@IMSA,  formerly  the  Center  for  Problem  Based 
Learning  at  the  Illinois  Math  and  Science  Academy.  The  second  was  the  scaffolding  model  for  open-ended  learning 
environments  described  by  Hannafin  et  al  (1999). 

The  design  process  began,  as  delineated  by  The  Center@IMSA,  by  identifying  conceptual  and  skill-based 
learning  outcomes  as  well  as  the  significant  characteristics  of  the  learners.  For  this  study,  the  Arizona  State 
Department  of  Education  Curriculum  Standards  provided  specific  learning  outcomes  for  sixth  and  seventh  grade 
students.  The  next  step  was  to  identify  an  ill-structured,  complex  problem  based  on  a real-world  context  which 
would  provide  maximum  integrative  curricular  yield  and  learner  appeal.  The  problem  statement  created  for  this 
study  asks  students  to  adopt  the  role  of  adventurer  to  plan  a circumnavigation  of  the  earth  by  balloon,  a task 
addressing  science,  social  studies,  mathematics  and  English  learning  outcomes.  The  resulting  unit  was  named  Up, 
Up  & Away!.  The  remaining  steps  in  The  Center@IMSA’s  design  process  were  followed  to  identify  relevant 
information  resources,  develop  assessments  for  checking  learners’  understanding  throughout  the  unit  sequence,  and 
build  a teaching  and  learning  template  which  supports  learners’  thinking  throughout  the  inquiry  process  by 
structuring  student  problem  solving  through  the  following  seven  steps:  meet  the  problem,  identify  what  is  known 
about  the  problem  as  well  as  what  needs  to  be  known,  define  the  problem  statement,  gather  and  share  information, 
generate  possible  solutions,  evaluate  the  fit  of  possible  solutions,  and  present  the  best  fit  solution  in  the  form  of  a 
performance  assessment. 

While  implementation  of  The  Center@IMSA’s  guidelines  and  the  suggested  unit  template  provided  some 
measure  of  learner  and  teacher  support,  the  design  team  felt  that  additional  scaffolds  aimed  at  supporting  specific 
aspects  of  student  and  teacher  performance  would  likely  contribute  to  increased  achievment  and  attitudes  toward  the 
unit  on  the  part  of  both  students  and  teachers.  To  provide  more  detailed  scaffolding,  the  design  team  turned  to  the 
scaffolding  portion  of  the  Hannafin,  Land  and  Oliver  (1999)  model.  This  model  categorizes  scaffolds  as  one  of  four 
types:  conceptual,  metacognitive,  procedural  and  strategic. 

The  design  team  first  considered  scaffolds  for  students,  beginning  with  conceptual  scaffolds.  According  to 
the  Hannafin,  Land  and  Oliver  (1999)  model,  conceptual  scaffolds  guide  students  in  what  to  consider.  This 
guidance  may  take  such  forms  as  a graphical  advance  organizer  or  content  outline  showing  superordinate  and 
subordinate  relationships.  To  provide  scaffolding  of  this  type,  the  Up,  Up  & Away I student  interface  was  organized 
under  four  conceptual  headings:  Prior  Attempts,  Weather  & Geography,  News  Articles  and  Balloon  Design 
(Figure  1).  Under  each  heading  are  links  to  web  sites  relevant  to  solving  the  unit  problem. 


PrferAttanpte 


Mrwt  Articles 

* Kftund  ih* 


Figure  1.  Hypermedia  Student  Interface 
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Conceptual  scaffolds  may  also  recommend  certain  procedures  or  tools  at  particular  stages  of  the  problem 
solving  process.  Additional  student  conceptual  scaffolds  took  the  form  of  a variety  of  record  keeping  / data 


collection  forms  designed  to  support  completion  of  the  unit  problem.  Planning  a circumnavigation  of  the  earth  by 
balloon,  the  unit  problem,  was  divided  into  three  strands:  designing  a balloon,  writing  a travel  plan,  and  creating  a 
list  of  supplies.  Students  were  provided  a conceptual  scaffold  in  the  form  of  a data  collection  document  for  each  of 
these  strands.  For  example,  the  travel  plan  form  guided  students  in  their  consideration  of  take  off  and  landing 
points,  countries  to  be  crossed  or  avoided,  determining  the  projected  length  of  the  flight,  and  choosing  the  most 
suitable  hemisphere  for  the  flight  based  on  current  jet  stream  winds. 

The  second  type  of  scaffold  described  in  the  Hannafin  et  al  (1999)  model  is  metacognitive,  which  guides 
students  in  how  to  think  as  they  complete  a task.  Up,  Up  & Away ! provided  no  metacognitive  scaffolding  for 
students.  A third  type  of  scaffold,  according  to  the  Hannafin  et  al  (1999)  model,  is  procedural  scaffolds  which  guide 
students  in  how  best  to  utilize  the  features  of  an  open-ended  learning  environment  through  such  things  as  tutoring, 
pop-up  help,  or  some  other  form  of  job  aid.  The  design  team  felt  that  the  Up,  Up  & Away!  interface  was  simple 
enough  that  student  procedural  scaffolding  was  unnecessary  beyond  a teacher  directed  introduction  to  the  interface. 
Lastly,  the  Hannafin  et  al  (1999)  model  describes  strategic  scaffolds  as  guides  for  learners  in  analyzing  or 
approaching  a learning  task  or  problem.  The  design  team  made  a student  strategic  scaffold  available  for  each  of  the 
three  strands  of  the  unit  through  the  Resouces  button  on  the  Up,  Up  & Away!  interface.  Selecting  the  Resouces 
button  took  students  to  a menu  page  where  they  could  choose  an  appropriate  type  of  help  (Figure  3).  Selecting  a 
specific  help  topic  then  took  students  to  a strategic  scaffold  designed  to  support  completion  of  a given  strand  of  the 
problem  (Figure  4).  Students  could  also  download  the  previously  described  conceptual  scaffolds  from  these  pages. 
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Figure  2.  Resources  Menu  Page 
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Figure  3.  Balloon  Design  Strategic  Scaffold 


Once  the  design  team  felt  scaffolds  sufficient  to  accommodate  student  success  had  been  incorporated  into 
the  hypermedia  interface,  attention  was  turned  to  supporting  the  teacher.  Recognizing  that  the  role  of  the  teacher  in 
implementing  student-centered  PBL  instructional  methods  might  represent  an  unfamiliar  and  intimidating  situation, 
the  design  team  focused  on  scaffolding  as  many  instructional^  sound  strategies  in  the  unit’s  teacher  guide  as 
deemed  essential  for  teachers  to  be  effective.  Teacher  scaffolds  in  Up,  Up  &Away!  include  conceptual  and  strategic 
types. 
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Conceptual  teacher  scaffolds  guide  instructors  in  what  to  consider  at  particular  stages  of  the  unit  or  lesson 
presentation.  For  example,  according  to  The  Center@IMSA,  providing  student  feedback  on  progress  toward  solving 
the  unit  problem  represents  an  essential  teacher  behavior  in  PBL  environments.  To  support  both  this  understanding 
and  behavior,  the  design  team  provided  student  feedback  forms  for  each  of  the  three  strands  of  the  unit.  An 
additional  example  concerns  the  necessity  for  abandoning  the  familiar  role  of  information  provider  in  favor  of 
monitoring  student  progress  and  guiding  student  thinking.  To  support  teachers  in  their  role  as  facilitators  of  student 
knowledge  construction,  the  design  team  included  questions  designed  to  support  this  behavior. 

Teacher  strategic  scaffolds  serve  to  guide  in  analyzing  and  approaching  the  unit  implementation. 
According  to  Hannafin  et  al  (1999),  they  can  take  the  form  of  start-up  questions  or  advice  from  experts.  The  design 
team  provided  a variety  of  such  expert  advice  to  support  teachers  unfamiliar  with  PBL  teaching  techniques, 
including  such  things  as  how  to  generate  enthusiasm  for  the  unit,  group  students  for  instruction,  and  provide 
feedback. 

The  next  phase  in  the  development  process  was  to  perform  a formative  evaluation.  Two  trials  were 
conducted. 

Trial  1 

Subjects 

Participants  were  forty,  sixth  grade  students  enrolled  in  two  technology  classes  at  an  urban  middle  school 
and  an  instructor  in  his  first  year  of  teaching. 

Materials 

Materials  included  the  hypermedia-based  PBL  unit  Up,  Up  & Away!  and  the  Up,  Up  & Away ! teacher 

guide. 

Procedures 

The  learning  environment  consisted  of  both  a 25  station  computer  lab  and  the  teacher’s  home  room.  The 
PBL  unit  Up,  Up  &Away ! was  loaded  on  the  district  server  and  available  through  a browser  on  the  computers  in  the 
computer  lab.  Due  to  time  constraints,  only  two  strands  of  the  unit,  Balloon  Design  and  Supply  List  were 
implemented.  Students  worked  in  informal  cooperative  groups  of  two  to  three  for  a total  instructional  time  of 
approximately  eight  hours,  at  the  end  of  which  they  turned  in  their  final  projects.  Following  completion  of  the  two 
unit  strands,  students  completed  an  attitude  survey  while  the  teacher  participated  in  an  exit  interview. 

Data  Sources 

Data  concerning  implementation  of  the  unit  was  collected  through  independent  observation  by  two 
members  of  the  development  team. 

Achievement  was  measured  by  scoring  final  student  projects  according  to  grading  rubrics,  one  for  each 
strand  of  the  unit.  The  rubrics  assessed  quality  and  completeness  of  content.  Three  evaluators  scored  each  student 
project  independently.  Final  project  scores  were  calculated  by  averaging  the  three  independent  scores. 

Student  attitudes  were  measured  by  a eight  item  attitude  survey.  Five  Likert  scaled  items,  arranged  on  a 
four  point  scale  from  Strongly  Agree  to  Strongly  Disagree,  asked,  about  such  things  as  the  ease  of  using  the  student 
interface,  student  effort  in  completing  the  unit,  ease  of  finding  information  and  feelings  toward  Up,  Up  & Away!. 
The  attitude  survey  also  included  three  free  response  items  soliciting  student  opinions  on  what  they  liked  best  about 
the  unit,  what  would  have  helped  them  do  a better  job  of  completing  the  unit,  and  what  they  felt  should  be  changed 
about  the  unit. 

Teacher  attitudes  were  collected  through  an  exit  interview  which  solicited  a broad  spectrum  of  feelings  and 
responses  toward  the  Up,  Up  & Away!  unit  and  teacher  guide. 

Results 

Observation  data  revealed  student  inattention  and  confusion  in  knowing  how  to  proceed.  Student  scaffolds 
available  through  the  interface  weren’t  used.  The  teacher  didn’t  support  use  of  the  scaffolds  and  spent  a large 
amount  of  time  trouble-shooting  technical  problems  and  tracking  due  dates  for  student  projects.  As  a result,  he  was 
frequently  disengaged  from  the  students’  learning  process. 

Achievement  as  measured  by  mean  final  group  project  scores  was  45%  for  the  Balloon  Design  and  10%  for 
the  Supply  List. 

Mean  student  attitude  scores  are  shown  in  Table  1.  Item  one  indicated  that  the  majority  of  students  felt 
working  on 
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Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 

1 . Working  on  the  Up,  Up  & Away!  project  was  fun. 

17% 

48% 

20% 

15  % 

2.  It  was  easy  to  find  the  information  I needed  in 
order  to  complete  the  assignment. 

20% 

43  % 

10% 

27  % 

3.  I learned  a lot  while  completing  the  Up,  Up  & 
Away!  assignment. 

15% 

43  % 

27  % 

15  % 

4.  I worked  hard  on  the  Up,  Up  & Away! 
assignment. 

40% 

37% 

13  % 

10% 

5.  I would  enjoy  working  on  another  project  like 
Up,  Up  & Away!. 

22% 

22% 

18  % 

38  % 

Table  l.  Trial  1 Mean  Student  Attitude  Scores 


Up,  Up  & Away ! was  fun.  Item  two  revealed  that  a total  of  approximately  40%  either  disagreed  or  strongly 
disagreed  that  finding  information  necessary  for  solving  the  problem  was  easy.  For  the  third  item,  over  half  the 
students  felt  they  learned  a lot  while  working  on  the  project,  while  on  item  four,  almost  80%  either  agreed  or 
strongly  agreed  that  they  worked  hard  on  the  unit.  Finally,  while  44%  agreed  they  would  enjoy  working  on  a unit 
like  this  again,  56%  felt  they  would  not. 

Teacher  interview  data  included  an  acknowledgment  that  he  hadn’t  fully  read  nor  used  the  teacher’s  guide. 
The  teacher  also  recognized  that  he  hadn’t  supported  student  use  of  the  scaffolds  available  through  the  interface. 
The  teacher  confirmed  that  many  students  were  inattentive  during  the  introductory  activities  and  suggested  that  the 
reading  level  of  the  introductory  articles  may  have  been  too  high.  Despite  these  difficulties,  the  teacher  expressed 
positive  attitudes  toward  the  unit  overall  and  an  intention  to  teach  it  again,  this  time  with  greater  attention  paid  to 
reading  and  following  the  suggested  instructional  strategies  in  the  teacher’s  guide. 

Discussion 

There  were  several  limitations  to  the  first  trial,  among  them  the  fact  that  only  two  of  the  three  unit  strands 
were  completed  and  that  the  teacher  neglected  to  fully  implement  the  strategies  contained  in  the  teacher’s  guide. 
Combined  results  from  the  various  data  sources  indicated  that  students  were  largely  unsuccessful  in  attaining  the 
unit  goals.  Nevertheless,  results  of  Trial  ldid  suggest  means  for  refining  the  unit,  as  observation  had  revealed  that 
without  teacher  guidance,  students  failed  to  explore  the  Help  button  with  the  result  that  many  of  the  scaffolds 
available  weren’t  utilized.  The  developers  suspected  that  labeling  the  button  “Help”  was  ineffective,  as  the  common 
interpretation  of  help  is  generally  software  assistance,  rather  than  content  or  learning  assistance.  Accordingly,  the 
button  was  relabeled  as  “Resources”.  To  address  teacher  concerns  that  the  unit’s  introductory  articles  were  too 
difficult,  they  were  edited  for  readability  and  additional  comprehension  questions  were  provided  to  support  student 
engagement.  Following  these  changes,  a second  trial  was  initiated. 

Trial  Two 

Participants 

Sixteen  students  enrolled  in  a sixth  grade  gifted  and  talented  program  at  a metropolitan  elementary  school 
in  a large,  southwestern  city  participated  in  Trial  2.  The  teacher  was  an  18  year  veteran  experienced  in  student- 
centered  teaching  techniques. 

Materials 

Materials  included  the  revised  versions  of  the  Up,  Up  & Away ! unit  and  teacher  guide. 

Procedures 

The  learning  environment  consisted  of  both  a 27  station  computer  lab  and  the  teacher’s  home  room  where  a 
single  computer  was  available.  Up,  Up  & Away!  was  loaded  on  the  district  server  and  available  through  all  the 
computers.  All  three  strands  of  the  unit  were  implemented  over  a period  of  four  weeks.  Students  worked  in 
informal  cooperative  groups  of  three  to  four  for  a total  instructional  time  of  approximately  20  hours.  Upon 
completion  of  the  unit,  students  presented  their  final  projects  during  an  evening  assembly  for  parents.  Following 
completion  of  the  unit,  students  completed  an  attitude  survey  while  the  teacher  participated  in  an  exit  interview. 


Data  Sources 

Data  sources  for  this  trial  were  identical  to  the  first  implementation. 

Results 

During  observations  of  Trial  2,  the  development  team  noted  that  the  teacher  used  a variety  of  successful 
strategies  such  as  actively  focusing  student  activities  and  monitoring  their  progress.  As  a result,  students  appeared 
organized,  aware  of  what  to  do,  and  confident  of  how  to  do  it.  Student  use  of  the  scaffolds  available  through  the 
Resources  button  was  limited,  with  most  students  opting  to  make  notes  on  their  own  paper. 

Results  for  student  achievement  revealed  an  overall  mean  score  of  97.5%  on  the  final  projects  as 
determined  by  grading  according  to  the  unit  assessment  rubrics.  Mean  student  attitude  survey  results  were  largely 
positive  (Table  2).  With  a ranking  of  strongly  agree  or  agree,  100  percent  of  students  felt  that  working  on  the  unit 
was  fun,  they  learned  a lot  while  completing  the  unit,  and  they  would  enjoy  working  on  another  project  like  “Up, 
Up,  & Away!” 




Strongly 
Agree  * , 

Agree 

Disagree 

Strongly^ 
Disagree,  ^ 

1 . Working  on  the  Up , Up  & Away!  project  was  fun. 

71  % 

29% 

2.  It  was  easy  to  find  the  information  I needed  in  order 
to  complete  the  assignment. 

8% 

54% 

38% 

3.  I learned  a lot  while  completing  the  Up,  Up  & 
Away!  assignment. 

71  % 

29% 

4.  I worked  hard  on  the  Up,  Up  & Away!  assignment. 

64% 

36% 

5.  I would  enjoy  working  on  another  project  like  Up, 
Up  & Away!. 

86% 

14% 

Table  2.  Trial  2 Mean  Student  Attitude  Scores 


Ninety-two  percent  of  students  strongly  agreed  or  agreed  that  using  the  hypermedia  website  was  easy.  However,  40 
percent  of  students  disagreed  with  the  statement  that  it  was  easy  to  find  information  to  complete  the  assignment. 

Results  of  the  teacher  exit  interview  revealed  a positive  experience.  The  teacher  indicated  satisfaction  with 
the  learning  students  demonstrated  and  an  intention  to  teach  the  unit  again,  although  she  stated  a desire  for 
individual  achievement  data  as  a means  for  measuring  personal  student  knowledge  gains.  The  teacher  also  reported 
she  had  read  the  teacher’s  guide  carefully  and  that  the  lesson  design  structure,  as  well  as  the  teacher  scaffolds, 
supported  her  presentation  of  content  with  which  she  had  little  experience  or  confidence.  She  ended  by  suggesting 
additional  scaffolds  that  might  further  assist  teachers. 

Discussion 

Combined  results  of  the  various  data  sources  in  Trial  2 indicated  students  were  generally  successful  in 
attaining  the  objectives  in  all  three  unit  strands  without  making  significant  use  of  the  scaffolds  available  through  the 
Resources  button.  Student  success  may  have  been  supported  by  the  extensive  use  of  teacher  scaffolds  designed  to 
motivate  and  focus  student  attention  and  activity  employed  by  the  teacher.  As  the  tryout  population  was  composed 
of  students  identified  as  gifted  and  talented,  however,  it’s  not  possible  to  generalize  their  success  to  sixth  grade 
students  at  large. 

Based  on  observations  and  teacher  input,  additional  changes  were  made  to  the  unit  following  Trial  2.  In 
terms  of  student  scaffolds,  a self-assessment  form  was  added  to  assist  students  in  evaluating  their  own  progress. 
Also  added  was  a Final  Presentation  Assignment  Form  detailing  presentation  expectations  to  encourage  teachers  to 
have  students  actively  present  their  final  projects  rather  than  simply  handing  them  in.  Along  with  the  Final 
Presentation  Assignment  Form,  a Presentation  Feedback  Checklist  was  added  to  support  both  the  teacher  and 
students  with  giving  feedback  following  final  project  presentations.  The  goal  was  not  only  to  encourage  teachers  to 
require  student  presentations,  but  to  support  students  in  actively  listening  to  their  peers’  presentations.  Additional 
teacher  scaffolds  added  to  the  unit  included  a Student  Progress  Report  form  designed  to  support  teachers  in 
monitoring  student  progress  and  providing  feedback.  Also  added  was  a unit  review  lesson,  unit  review  question 
bank,  and  an  objective  posttest  to  allow  measurement  of  individual  student  achievement. 
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Implications 

This  study  suggests  that  hypermedia-based  PBL  may  be  effective  for  gifted  elementary  students.  It  also 
suggests  that  success  in  this  environment  may  be  related  to  teacher  experience  with  student-centered  learning 
techniques.  Student  success  may  also  relate  to  the  depth  and  complexity  of  web  sites  included  as  information 
resources.  Observational  data  of  student  behaviors,  as  well  as  attitude  data  showing  that  approximately  40  percent 
of  students  in  both  trials  found  locating  information  on  the  web  sites  difficult,  suggests  that  linking  more  directly  to 
web  site  pages  containing  relevant  information  rather  than  to  a web  site’s  introductory  screen  may  better  support 
student  success. 

In  terms  of  teacher  and  student  scaffolds  and  their  effectiveness  in  a hypermedia-based  PBL  environment, 
this  study  suggests  that  while  student  scaffolds  may  not  be  necessary  for  student  success,  teacher  scaffolds  appear  to 
be  important  for  generating  student  interest  and  supporting  teacher  confidence.  For  example,  the  scaffolds  providing 
expert  advice  for  implementing  the  unit  suggest  a bulletin  board  as  one  means  for  generating  interest.  In  Trial  2, 
when  the  teacher  created  a bulletin  board  as  well  as  a Parent  Involvement  Newsletter  inviting  parents  to  a special 
end-of-unit  presentation  event,  student  interest  was  high.  Additionally,  the  teacher  indicated  that  the  scaffolds  in  the 
teacher  guide  supporting  presentation  of  the  unit  boosted  her  confidence  in  knowing  how  to  begin  the  unit  and 
engage  students  in  a PBL  environment. 

Further  Research 

To  fully  address  the  suitability  of  hypermedia-based  PBL  in  the  upper  elementary  grades  requires 
additional  study  with  subjects  not  identified  as  gifted  and  talented.  To  better  measure  unit  effectiveness  through 
individual  learning  gains,  these  students  should  be  assessed  not  only  on  the  basis  of  a group  project,  but  with  an 
objective  based,  individual  posttest  as  well. 
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Abstract 

This  paper  presents  results  of  an  intensive  comparative  case  study  of  five  colleges  and  universities  using 
distance  education . The  study , through  interviews  with  the  president  and  other  senior  administrators  and  focus 
groups  with  faculty  and  administrators  involved  in  distance  education,  found  that  issues  related  to  faculty  and  other 
personnel,  marketing,  competition,  budget,  and  planning  were  more  significant  for  the  success  of  distance  education 
than  technological  issues.  Implications  for  planning  and  implementing  successful  distance  programs  are  presented. 

Opportunities  and  Challenges  of  Distance  Learning 

Distance  learning  technologies  offer  many  new  opportunities  for  institutions  of  higher  education.  These 
opportunities  include:  (a)  facilitating  improvements  in  the  teaching/leaming  process;  (b)  expanding  the  geographic 
reach  of  an  institution’s  programs;  and  (c)  facilitating  more  effective  service  of  the  student  population,  e.g.,  offer 
convenience,  greater  course  selection,  and  academically  richer  programs  (Smith,  1998). 

While  educational  technologies  are  a means  to  facilitate  learning,  distance  learning  is  more  than  just  an 
instructional  delivery  medium.  Distance  education  represents  a significant  change  from  traditional  instruction 
(Means,  1994).  Distance  education  breaks  from  traditional  instruction  by  changing  the  social  dynamics  of  education, 
bringing  the  school  to  the  student  (Holloway  and  Ohler,  1995).  Traditional  classroom  models  may  not  provide  the 
best  models  for  the  distance  classroom  because  distance  learning  is  fundamentally  different.  Instructors  cannot 
simply  switch  from  traditional  to  distance  delivery  systems;  instead,  substantial  planning  and  organization  is 
required  to  teach  in  a distance  learning  environment  (Martin  & Bramble,  1996;  Gibson  & Gibson,  1995).  Further, 
students  need  an  orientation  to  strategies  of  how  to  learn  effectively  in  a distance  learning  environment  (Martin  & 
Bramble). 

Typically,  many  institutions  begin  a distance  education  initiative  encouraged  by  the  many  potential 
benefits,  influenced  by  their  competition,  and  prompted  by  the  fear  of  not  being  involved  in  distance  education. 
Colleges  and  universities  report  entering  into  a distance  education  initiative  without  sufficient  planning,  without  a 
clear  mission  for  the  program,  and  without  realizing  many  of  the  challenges  that  will  surface  as  they  conduct  their 
program. 

Description  of  the  Study 

This  paper  presents  the  results  of  a research  study  conducted  in  fall,  1999  to  examine  the  challenges  facing 
higher  educational  institutions  in  migrating  to  distance  learning.  The  study  consisted  of  five  case  studies  conducted 
at  higher  educational  institutions  in  Pennsylvania.  Two  institutions  were  large  and  three  were  small;  two  were  public 
and  three  were  private.  Four  of  the  five  institutions  had  been  identified  as  having  considerable  experience  in  distance 
learning. 

At  each  institution  the  researchers  interviewed  the  president  and  other  senior  administrators  and  conducted 
a focus  group  with  administrators  and  key  faculty  involved  in  distance  education.  Participants  in  the  studied 
reviewed  and  analyzed  a comprehensive  model  developed  by  the  researchers  for  applying  principles  of 
organizational  agility  to  distance  learning. 

Data  from  the  individual  and  group  interviews  were  analyzed  using  standard  procedures  for  the  analysis  of 
case  study  data.  The  researchers  investigated  how  institutions  of  higher  education  could  strategically  coordinate 
human  resources,  organizational  dynamics,  and  distance  learning  technologies  in  a systematic  way  in  order  to  gain 
and  maintain  a competitive  edge  in  the  educational  market  place,  and  in  order  to  be  more  responsive  to  students. 
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Major  Findings  of  the  Study 

While  many  educational  administrators  focus  on  technological  hardware  and  software  issues,  they  may 
overlook  more  important  issues  that  are  integral  to  success  of  their  institution’s  distance  learning  efforts.  The  authors 
found  that  the  institutions  studied  considered  technological  issues  to  be  of  lower  priority  in  designing  and 
conducting  an  effective  distance  learning  program.  The  more  significant  factors  that  emerged  as  vital  to  the  success 
of  a distance  education  program  involved  human  factors,  specifically:  interpersonal  dynamics,  attitudes, 
organizational  culture,  styles  of  management,  and  styles  of  communication. 

The  following  categories  of  concern  in  migrating  to  offering  distance  education  emerged  from  the  research 

study: 

1 . faculty  issues; 

2.  academic  issues; 

3.  marketing/competition; 

4.  budget/fiscal; 

5.  planning; 

6.  personnel  issues; 

7.  technological  equipment. 

Each  of  these  categories  of  concern  will  be  examined  in  more  detail  in  the  following  sections.  Implications  for 
planning  and  implementing  successful  distance  programs  will  be  presented  and  discussed. 

Faculty  Issues 

Faculty  issues  emerged  as  the  major  challenge  facing  institutions  in  migrating  to  offering  distance 
education.  Many  faculty  members  joined  their  institutions  without  experience  or  knowledge  in  technology.  Fearing 
changes  in  working  conditions  and  potential  loss  of  job  security,  many  faculty  resist  learning  about  or  using  distance 
technologies.  Some  faculty  members  stated  that  they  would  retire  rather  than  use  technology  in  their  classroom. 
Administrators  found  that  they  must  work  to  educate  faculty  and  to  encourage  them  to  participate  in  distance 
education.  Overcoming  faculty  resistance  to  distance  learning  emerged  as  the  major  issue  in  conducting  a distance 
learning  program.  For  example,  faculty  resistance  is  visible  in  the  influence  of  the  faculty  union  within  public 
institutions.  Faculty  unions  complicate  the  course  approval  process  for  distance  courses,  pressure  the  faculty  to  not 
participate  in  distance  learning  activities,  and  lobby  for  significantly  increasing  financial  remuneration  for  faculty 
participating  in  distance  courses. 

Academic  Issues 

Faculty  members  mentioned  many  academic  challenges  to  offering  distance  courses.  Among  these  were  the 
challenges  of: 

1 . intellectual  property  rights; 

2.  having  distance  courses  pass  through  the  course  approval  process; 

3.  exam  honesty  for  remote  sites; 

4.  providing  an  adequate  help  desk  to  support  students; 

5.  the  need  to  provide  orientation  programs  for  distance  learning  students; 

6.  the  need  to  address  residency  requirements  for  distance  programs. 

Marketing  and  Competition  Issues 

Institutions  were  concerned  with  institutional  reputation  and  image  when  it  comes  to  offering  distance 
education.  They  realized  that  distance  education  will  expose  the  quality  of  their  program,  and  they  feared  that 
offering  a poor  quality  program  may  damage  their  reputation.  While  institutions  felt  pressure  to  get  involved  in 
distance  learning,  they  were  concerned  with  competing  in  a market  against  private,  for-profit  institutions  that 
exclusively  offer  distance  education.  Institutions  were  concerned  about  their  program  mission  and  about  specifically 
defining  their  niche  regarding  offering  distance  education. 

Budget  and  Fiscal  Issues 

All  of  the  institutions  mentioned  budget  issues  quite  frequently.  Issues  of  special  concern  were  ways  to 
fund  the  high  startup  costs  involved  in  distance  learning  and  the  funding  the  ongoing  costs  involved  in  offering 
distance  learning  programs.  The  need  to  use  scarce  resources  more  efficiently  is  prompting  all  of  the  institutions 
studied  to  identify  their  core  competencies  (those  areas  in  which  they  excel),  and  to  use  these  core  competencies  as 
the  basis  on  which  to  enter  the  distance  learning  arena.  Further,  in  order  to  function  more  efficiently  and  effectively 
in  an  increasingly  competitive  environment,  aft  of  the  institutions  presented  evidence  of  seeking  out  partnerships 
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with  other  institutions  where  it  enhanced  their  distance  learning  academic  programs.  All  reported  realizing 
considerable  benefits  from  partnering,  including  increased  efficiency,  enhanced  program  efforts,  and  positive 
experience  for  faculty. 

Planning 

Subjects  at  each  of  the  five  institutions  studied  reported  that  past  planning  for  technology  and  distance 
learning  was  insufficient.  Several  institutions  demonstrated  a need  for  assistance  in  long-range,  strategic  planning 
for  technology.  Subjects  from  two  institutions  indicated  failed  attempts  at  strategic  planning. 

Subjects  from  two  institutions  expressed  concern  over  their  inability  to  finalize  certain  key  decisions  in  this 
area.  Further,  subjects  from  two  of  the  small,  private  institutions  indicated  a need  to  make  definitive  decisions, 
which  if  not  made  would  seriously  impede  further  institutional  progress. 

Two  institutions  indicated  that  planning  receives  more  attention  after  an  institution  has  been  involved  in 
distance  learning  for  about  two  to  three  years,  after  personnel  become  more  aware  of  the  realities  of  distance 
learning  and  the  need  to  invest  in  more  conscientious  planning.  Although  evidence  of  basic  planning  regarding 
distance  learning  was  found  in  the  two  public  institutions,  as  stated  earlier,  evidence  of  more  careful,  conscientious 
planning  was  found  in  the  two  small,  private  institutions. 

Personnel  Issues 

In  addition  to  challenges  related  to  faculty,  challenges  related  to  hiring  and  dealing  with  non-faculty 
personnel  also  surfaced  in  the  study.  Problems  cited  included  difficulties  in  identifying  and  hiring  qualified  distance 
learning  coordinators,  which  are  related  to  the  problem  of  offering  competitive  salaries  for  distance  learning 
technical  personnel.  One  institution  reported  hired  a technology  director  who  was  ineffective  in  relating  to  and 
dealing  with  faculty.  This  person,  apparently  more  concerned  about  the  equipment  than  the  people  involved, 
prevented  the  faculty  from  practicing  and  using  technological  equipment,  and  set  back  the  institution’s  distance 
learning  efforts  significantly. 

Technological  Equipment 

One  definite  challenge  reported  by  all  five  institutions  studied  was  designing  and  implementing  distance 
learning  programs  is  selecting,  financing,  and  maintaining  technological  equipment.  The  selection  of  hardware  and 
software  also  impacts  on  staffing  decisions.  More  sophisticated  hardware  and  software  may  provide  greater 
capabilities  for  the  distance  learning  program,  but  the  potential  advantages  will  not  be  realized  if  the  additional 
competent  staff  support  required  and  advanced  faculty  training  are  not  provided  at  the  same  time.  If  the  institution  is 
planning  on  partnering  with  other  institutions,  than  decision  factors  in  the  selection  of  equipment  extend  outside  of 
the  institution  itself  to  its  partners,  in  order  to  ensure  compatibility. 

Implications  for  Planning  and  Implementing  Successful  Distance  Programs 

Analysis  of  the  data  collected  from  individual  and  group  interviews  with  faculty  and  administrators  at  the 
five  institutions  of  higher  education  studied  provides  support  for  a number  of  recommendations  for  effective  practice 
of  distance  learning  academic  programs.  These  recommendations  relate  to  faculty  and  staff  support,  program 
planning,  and  program  organization.  These  implications  will  be  discussed  below. 

Invest  In  and  Support  Faculty  and  Faculty  Initiatives 

Individuals  responsible  for  developing  distance  learning  programs  should  attempt  to  harness  the  wealth  of 
talent  and  expertise  already  available  at  their  institution.  Colleges  and  universities  should  invest  in  the  faculty  by 
providing  ample  training  opportunities,  by  capitalizing  on  the  interests  of  the  faculty,  and  by  providing  support  for 
faculty  initiatives.  The  institution  should  create  an  atmosphere  where  faculty  feel  safe  to  experiment  with  using 
distance  technologies  in  new  ways.  Faculty  should  be  encouraged  to  explore  how  to  take  advantage  of  the  unique 
capabilities  of  particular  technologies  to  improve  the  teaching/leaming  process. 

Initiate  Dialogue  with  Faculty 

Be  aware  that  overcoming  faculty  resistance  to  distance  learning  is  a major  challenge  in  conducting  a 
distance  learning  program.  Initiate  a dialogue  between  faculty  and  administrators  to  determine  and  address  the 
concerns  of  faculty  regarding  distance  education.  Be  sensitive  to  faculty  concerns  about  changes  in  working 
conditions  and  potential  loss  of  job  security. 

Educate  faculty  about  the  importance  of  a distance  learning  initiative  to  the  institution.  Involve  faculty  and 
support  staff  by  making  available:  (a)  information  that  describes  how  effectively  the  institution  is  meeting  its 
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mission;  (b)  information  about  strategic  imperatives  being  planned  and  initiatives  being  undertaken;  (c)  information 
about  the  challenges  and  problems  faced  by  the  institution;  and  (d)  information  about  the  institution’s  financial 
stability.  Help  facilitate  faculty’s  discovery  of  how  distance  learning  supports  the  institution’s  mission,  how  it  helps 
address  particular  institutional  challenges,  and  how  it  adds  to  the  stability  of  the  institution. 

Work  to  Overcome  Faculty  Resistance  and  Increase  Faculty  Buy-In 

Educate  the  faculty  about  the  benefits  of  using  technology  in  the  classroom.  Faculty  are  motivated  by  the 
ability  to  improve  the  teaching/learning  process.  Distance  technologies  hold  great  potential  to  enrich  the  classroom, 
to  connect  students  with  the  larger  research  community,  and  to  bring  authentic  examples  into  a lesson.  Provide 
training  opportunities  to  faculty  to  enlighten  them  about  the  benefits  of  using  distance  education  technologies  both 
for  themselves  (e.g.,  connecting  with  other  colleagues,  benefits  to  research)  and  for  their  students  (e.g.,  improved 
access  to  information  resources,  networking  within  a larger  community  of  scholars). 

Engage  in  More  Conscientious,  Focused  Planning 

Begin  by  defining  a clear  and  focused  mission  for  the  distance  education  program.  Capitalize  on  the  sense 
of  community  on  the  campus.  Open  a dialogue  between  faculty  and  administrators  to  discuss  the  various  issues, 
challenges,  and  implications  that  arise  in  offering  distance  education. 

Hire  Technologists  with  Strong  Interpersonal  Skills 

Technical  support  personnel  with  excellent  technological  knowledge  and  skills  who  do  not  also  have 
excellent  interpersonal  skills  will  not  be  able  to  support  faculty  in  learning  to  use  distance  learning  technologies 
effectively.  The  interpersonal  skills  required  include  in  particular  the  ability  to  empathize  with  novice  learners  and 
the  ability  to  explain  procedures  using  vocabulary  the  novice  learner  can  understand. 

Develop  a Shared  Vision 

Promote  open  discussion  about  distance  learning  programs  involving  all  levels  of  the  campus  community, 
including  faculty,  students,  and  support  staff.  Do  not  establish  a “top-down”  directive.  Instead,  work  cooperatively 
to  produce  a shared  vision  that  the  whole  campus  community  has  helped  to  develop.  This  approach  will  result  in  an 
increased  number  of  individuals  feeling  ownership  of  the  vision,  and  more  people  will  support  the  vision. 

Plan  and  Support  Migration  (Evolutionary  Change)  Vs.  Revolutionary  Change 

Evolutionary  change  is  less  threatening,  is  easier  to  implement,  and  is  easier  to  sustain.  Support  this 
migration  by  investing  in  faculty  training,  by  providing  sufficient  support  staff,  and  by  supporting  faculty  efforts. 

Integrate  Distance  Learning  Technologies  into  the  Overall  Academic  Program 

Instead  of  viewing  distance  education  as  a separate  entity,  treat  distance  learning  technologies  as  effective 
new  tools  to  enhance  the  teaching/learning  process.  Invest  in  support  staff,  equipment,  and  training  in  order  to  make 
the  atmosphere  conducive  for  faculty  to  integrate  these  technologies  into  classes  on  a regular  basis. 

Provide  Adequate  Support  Staff  for  Development  and  Delivery  of  Distance  Learning 

Invest  in  hiring  instructional  designers  and  other  technological  support  personnel  to  relieve  faculty  of 
dealing  with  the  technological  burdens  involved  in  offering  distance  learning  classes.  This  will  enable  the  faculty  to 
concentrate  on  substantive  issues  instead  of  becoming  bogged  down  in  technical  operational  details. 

Revise  Traditional  Policies  and  Practices  to  Support  Distance  Learning  Students 

Realize  that  distance  learning  is  fundamentally  different  from  learning  in  the  traditional  classroom. 
Consider  traditional  practices,  services,  and  policies  and  how  they  may  need  to  be  revised  to  accommodate  the  needs 
of  learners  and  instructors  participating  in  distance  education. 

Match  Distance  Learning  Technologies  to  Institutional  Conditions  and  Needs 

Select  distance  delivery  systems  that  provide  an  appropriate  fit  with  the  institution’s  mission  and  context. 
Before  selecting  delivery  systems,  ensure  that  the  medium  selected  will  effectively  and  efficiently  address  the  needs 
of  the  institution. 
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Emphasize  the  Role  of  Distance  Learning  Technologies  in  Promoting  Academic  Goals 

In  addressing  the  faculty,  approach  distance  learning  as  a tool  with  the  potential  to  improve  the 
teaching/leaming  process.  Do  not  approach  it  as  a way  to  primarily  advance  administrative  goals  without  regard  to 
academics. 

Elements  of  Successful  Distance  Learning  Programs 

A further  review  and  analysis  of  the  data  collected  from  the  five  institutions  studied  revealed  a number  of 
essential  elements  that  were  shared  by  all  of  the  distance  learning  programs  that  seemed  to  be  positioned  for  future 
success  in  offering  productive  distance  learning  programs: 

1.  Administrators  opened  a dialogue  that  involved  the  whole  college  community  in  planning  and  in 
developing  a vision  of  distance  learning. 

2.  Administrators  were  sensitive  to  the  concerns  of  faculty.  They  addressed  the  concerns  of  employment 
conditions,  job  security,  compensation,  and  the  faculty’s  fear  of  learning  and  using  new  technologies. 

3.  Administrators  committed  resources  to  invest  in  the  faculty.  They  sought  to  educate  the  faculty  as  to 
the  benefits  of  using  technologies,  provided  training  opportunities,  provided  support  for  faculty  in 
learning  and  using  technology,  and  encouraged  faculty  exploration  and  experimentation. 

4.  The  institutions  attempted  to  integrate  technologies  into  the  traditional  classroom  to  improve  the 
teaching/leaming  process,  rather  than  introducing  new  technologies  only  for  distance  learning 
programs.  Distance  learning  was  presented  as  an  extension  of  the  use  of  technology  to  promote 
teaching  and  learning,  rather  that  a means  to  produce  mass  education  more  economically,  to  increase 
revenues,  or  to  increase  market  share. 

5.  Administrators  and  faculty  realized  that  distance  learning  was  fundamentally  different  from  the 
traditional  classroom  model,  and  would  thus  require  a reconsideration  of  various  traditional  practices, 
policies,  and  working  conditions.  At  the  least,  they  openly  dealt  with  these  issues  as  they  arose.  At 
best,  they  initiated  steps  to  determine  in  advance  what  these  changes  were  and  how  they  could  best  be 
addressed. 

Conclusion 

While  educational  technologies  are  a means  to  facilitate  learning,  distance  education  is  more  than  just  an 
instructional  delivery  medium.  In  the  process  of  offering  distance  education,  the  institutions  studied  either 
discovered  or  were  beginning  to  discover  that  distance  learning  is  fundamentally  different  from  traditional 
instruction  in  the  following  ways: 

• Distance  learning  involves  addressing  changes  in  historical  practice  and  policy  that  have  to  date 
adequately  supported  the  traditional  classroom  model. 

• An  effective  distance  learning  program,  by  contrast,  requires  the  application  of  new  models  of 
instruction,  new  instructional  methodologies  and  skills,  and  new  ways  to  learn  effectively. 

• Distance  learning  also  involves  a change  in  the  ways  students  are  served,  the  ways  academic  programs 
are  marketed,  and  the  ways  in  which  instruction  is  organized  and  delivered. 

In  this  new  environment,  educators  have  had  to  address  many  issues,  including  the  following:  (a)  faculty 
compensation;  (b)  models  of  office  hours  and  other  faculty  responsibilities;  (c)  intellectual  property  rights;  (d) 
supporting  students  who  rarely  if  ever  visit  the  campus.  Further,  distance  learning  provides  a vehicle  for  responding 
to  the  educational  needs  and  demands  of  an  information  society  by  allowing  instruction  to:  (a)  change  from  a focus 
on  teaching  to  a focus  on  learning;  (b)  change  from  memorizing  information  to  harnessing  it;  (c)  become  more 
student-centered;  (d)  utilize  authentic  forms  of  assessment;  (e)  evolve  beyond  using  the  same  instructional  strategy 
for  all  students.  The  researchers  hope  that  by  exploring  and  exposing  some  of  the  challenges  institutions  face  in 
planning,  developing,  and  implementing  academic  distance  learning  programs,  their  research  will  help  educational 
professionals  realize  that  the  problems  they  face  are  common  to  all  institutions.  It  is  hoped  that  the  findings  of  this 
study,  and  the  implications  of  those  findings,  will  improve  the  practice  of  distance  learning  in  higher  education. 
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Abstract 

This  paper  reports  the  findings  of  a comparative  study  of  the  content  of  electronic  exchanges  among 
members  of  learning  communities  to  determine  what  kinds  of  online  dialogue  differences— if  any— exist.  The  virtual 
learning  communities  compared  are  two  cohorts  within  the  same  professional  preparation  program  for  teachers 
aspiring  to  become  school  leaders.  The  content  area  and  instructor  are  the  same  for  both  cohorts;  the  significant 
variable  is  the  mode  of  instructional  delivery.  One  cohort  received  nearly  all  instruction  through  a distance- 
learning program.  The  other  cohort  meets  together  weekly  and  uses  online  activities  for  approximately  30  percent 
of  instructional  delivery.  Comparative  data  used  are  online  messages  generated  during  the  first  semester  of  both 
programs  in  which  leadership  was  the  central  focus. 

Review  of  the  Literature 

Nearly  60%  of  the  nation's  higher  educational  institutions  use  information  technology  as  a mode  of 
instructional  delivery  (Lewis,  Snow,  Farris,  & Levin,  1999).  As  universities  increasingly  use  online  learning 
networks,  it  becomes  important  to  understand  how  online  interactions  support  academic  success  (Cox,  1999). 
Online  environments  can  enhance  learning  by  shifting  from  a teacher-centered  model  in  which  the  instructor  is  the 
source  of  knowledge  to  a learner-centered  model  in  which  peer  support,  interaction,  and  collaboration  are 
emphasized  (Beller  & Or,  1998;  Harasim,  1990;  Lebow,  1993).  Participants  communicating  within  virtual 
environments  use  asynchronous  dialogue  to  reflect,  debate,  critique,  give  feedback,  question,  answer,  and  engage  in 
multiple  other  communicative  behaviors  (Henri,  1992;  Mason,  1994).  Because  individuals  interacting  within  a 
virtual  setting  share  a range  of  intellectual  responses  similar  to  those  in  other  modes  of  communication  (Paccagnella, 
1997),  online  learning  activities  have  potential  to  provide  rich  opportunities  for  learning. 

Skeptics  of  cyber-learning  warn  that  it  offers  a potentially  false  or  artificial  sense  of  learning  that 
diminishes  individual  capacity  and  leads  to  fragmentation  (Heim,  1993).  Proponents  of  instructional  delivery 
through  technology  maintain  that  computers  promote  learning  because  ample  time  for  quality  feedback  is  available, 
provide  a buffer  for  those  less  adept  in  face-to-face  communications,  and  enhance  spontaneity  (Bresler,  1 990).  Some 
findings  suggest  that  learning  in  virtual  communities  increases  creative  flow  and  collaborative  possibilities  (Green, 
1996;  Knox-Quinn,  1993).  Because  research  about  the  implementation  of  successful  online  environments  for 
learning  is  a new  and  evolving  area  (Cox,  1999),  many  questions  remain  unanswered. 

The  ALPS  Program:  Cohorts  in  Professional  Preparation 

The  Administrative  Leadership  and  Policy  Studies  (ALPS)  division  within  the  School  of  Education  at  the 
University  of  Colorado  at  Denver  (UCD)  is  authorized  by  the  State  Board  of  Education  to  offer  training  for  aspiring 
school  principals  and  administrators  (Colorado  Department  of  Education  [CDE],  1997).  Following  the  state 
adoption  of  professional  standards  in  1994,  the  ALPS  faculty  progressively  revised  its  leadership  education  program 
into  a problem-based  (Ford,  Martin,  Muth,  & Steinbrecher,  1997;  Muth,  forthcoming),  active-learning  (Muth,  1999), 
portfolio-assessed  (Muth,  Murphy,  Martin,  & Sanders,  1996)  model.  The  leadership  preparation  program 
transformed  from  a series  of  on-campus  courses  into  unique  off-campus  cohorts  developed  through  school  district 
partnerships.  Because  most  ALPS  cohorts  are  developed  in  partnership  with  local  school  districts,  unique  problems 
of  practice  emerge  as  potential  projects  and  learning  events  (Martin,  Ford,  Murphy,  & Muth,  1998).  Partnership 
cohort  sessions  are  held  at  district  sites  and  jointly  taught  by  university  professors  and  administrative  practitioners. 

As  a standards-driven  program  (Ford,  Martin,  Murphy,  & Muth,  1996;  Murphy,  Martin,  & Muth,  1994),  the 
goal  is  to  endorse  graduates  as  competent  professionals  ready  to  assume  roles  as  school  leaders.  The  program  is  a 
sequence  of  four  learning  domains  that  concentrate  on  specific  areas  of  school  administration  and  connect  to 
concurrent  field  internships.  Individual  and  group  activities  within  the  domains  center  on  four  broad  topics: 
(a)leadership,  (b)  school  environment,  (c)  supervision  of  curriculum  and  instruction,  and  (d)  school  improvement. 
While  each  of  the  domains  has  an  integrated  set  of  field  activities  to  connect  content  to  practice,  a 135  clock-hour 
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intensive  internship  provides  additional  immersion  in  practice  and  experience  as  an  administrator.  Thus,  content 
learning  is  balanced  with  field  experiences  so  that  students  gain  clinical  skill  to  recognize  and  solve  problems  of 
professional  practice. 

The  adoption  of  a sophisticated  online  communication  system  by  the  UCD  School  of  Education  opened 
myriad  opportunities  to  integrate  online  instruction  and  learning  into  the  school's  licensing  programs.  The  FirstClass 
Client  e-mail  and  conferencing  system,  sold  by  SoftArc  and  dubbed  Colorado  Educators  Online  (CEO),  provides 
statewide  service  to  the  school,  area  districts,  and  educational  associations.  CEO  "permits  synchronous  as  well  as 
asynchronous  communications,  easy  file  sharing,  and  Internet  access"  (Muth,  2000,  p.  60)  and  allows  creation  of 
discussion  sites  known  as  conferences.  Within  a cohort  conference,  participants  can  post  questions,  comments,  and 
responses  viewed  by  all  conference  members.  Subconferences  facilitate  completion  of  special  online  projects. 

Same  Program,  Two  Different  Cohorts 

The  two  cohorts  within  the  ALPS  licensing  program  used  in  this  comparative  study  are  uniquely  different. 
Common  elements  made  it  possible  to  link  the  two  case  studies  for  purposes  of  comparison  of  electronic  exchanges: 
The  same  instructor  taught  the  two  cohorts  during  the  same  domain  of  the  ALPS  program.  Within  each  of  the  CEO 
cohort  conferences,  it  was  easy  to  identity  for  analysis  students'  online  messages  within  the  leadership 
subconference.  Hence,  the  student  online  exchanges  within  each  of  the  cohort  leadership  subconferences  in  the 
CEO  became  the  main  data  source  link  for  this  comparison 

Virtual  Learning  Community:  Distance  Learning  Cohort 

After  intensive  curriculum  refinement,  the  ALPS  division  launched  its  first  DL  principal  licensing  cohort 
for  students  who  live  in  areas  of  Colorado  remote  from  any  graduate  licensing  programs.  Participants  met  together 
as  a united  cohort  group  only  twice.  The  first  cohort  meeting  was  an  eight-day  orientation  on  the  UCD  campus  in 
June  1999.  An  important  reason  for  the  face-to-face  interactions  was  to  "build  a shared  culture  for  a strong  cohort 
and  effective  cluster  groups"  (Muth,  2000,  p.  65).  During  the  orientation,  students  selected  membership  to  small 
support  (cluster)  groups  that  were  determined  by  geographic  locale  and  that  met  at  the  discretion  of  the  members 
throughout  the  yearlong  program.  Significant  blocks  of  time  during  the  orientation  session  were  used  to  provide 
students  with  instruction  and  practice  in  using  CEO  (Muth,  2000).  The  DL  cohort  returned  to  UCD  in  July  2000  for 
its  second  cohort-wide  session.  During  those  two  days,  students  participated  in  reviews  for  the  required  state 
examination,  portfolio  defenses,  program  evaluation,  and  celebration  ceremonies  for  the  closing  of  the  cohort. 

Almost  all  instruction  and  group  discussion  in  the  DL  cohort  was  conducted  electronically  through  CEO. 
Sources  of  online  interaction  data  available  include  (a)  open  cohort  conference  discussions  and  (b)  chat  room 
discussions.  Personal  communications  not  sent  to  the  cohort’s  conference  were  not  accessible.  Additionally,  course 
syllabi  and  calendars,  student  responses  to  mid-program  telephone  interviews,  and  student  reflections  recorded 
during  exit  interviews  provide  additional  data  sources. 

Onsite  Learning  Community:  Empowerment  for  Change  in  Urban  Schools  Cohort 

A university-district  partnership  was  formed  in  a community  within  a short  distance  of  downtown  Denver. 
Like  other  urban  school  districts,  the  district  faces  multiple  challenges  beyond  alleviating  its  current  shortage  of 
potential  school  principals  and  administrators:  (a)  state-mandated  accountability  measures,  (b)  political  and  public 
scrutiny  of  student  scores  and  school  report  cards,  (c)  limited  resources  for  implementing  school  renewal  measures, 
and  (d)  a poorly  developed,  economically-stressed  community  support  base.  The  name  for  this  learning  cohort, 
Empowerment  for  Change  in  Urban  Schools,  embodies  the  vision  of  its  cohort  leader  for  expanding  leadership 
education  and  skills  development  into  forms  of  collaborative  leadership  processes  (Napier  & Lowry,  1999). 

As  in  other  ALPS  university-district  partnership  cohorts , approximately  one-third  of  the  curriculum 
instruction  in  the  ECUS  cohort  is  delivered  online.  Consistent  with  the  DL  cohort,  sources  of  online  interaction 
data  in  the  ECUS  cohort  conference  include  the  open  cohort  conference  discussions.  Data  collected  through  review 
of  artifacts,  responses  to  a questionnaire  about  CEO  use,  and  researcher  observation  of  cohort  sessions  provide 
additional  data  sources . A larger  case  study  of  the  ECUS  cohort  is  nearing  the  close  of  a year  of  data  collection. 

Strategies  for  Comparative  Analysis 

Since  the  two  cohorts  were  in  the  same  licensure  program  and  lead  by  the  same  instructor,  we  assumed  that 
the  content  and  instructional  activities  within  the  leadership  domains  of  the  distance-learning  cohort  and  the  onsite 
cohort  were  similar  in  message  substance.  Therefore,  we  developed  a coding  system  to  analyze  the  posted  messages 
in  the  subconference  labeled  by  the  course  number  for  the  leadership  domain.  Both  the  coding  and  analyses  were 
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conducted  manually.  Where  needed  for  further  understanding,  analysis  also  included  review  of  cohort  syllabi  and 
assignment  calendars,  observation  field  notes  of  the  ECUS  cohort,  and  student  responses  to  questionnaires. 

Coding  Message  Types 

The  display  format  of  messages  within  CEO  allows  a viewer  to  discern  quickly  the  differences  between 
threaded  discussions  (groups  of  response  messages  linked  to  an  initiated  message  by  a common  subject  heading)  and 
non-responded  messages  (to  which  no  written  reply  was  sent  to  the  conference).  Each  of  us  began  at  the  first 
message  posted  in  the  leadership  subconference  and  classified  messages  either  as  a non-responded  message  (M)  or 
as  a group  of  messages  within  a threaded  discussion  (T).  All  files  within  a threaded  discussion  were  chunked 
together  and  coded  as  one  threaded  discussion.  Messages  within  each  category  were  numbered  in  chronological 
order  by  date  beginning  with  the  first  message  to  appear  in  the  conference.  For  messages  that  evolved  into  threaded 
discussions,  we  classified  the  first  message  as  an  initiated  message  (I).  All  response  messages,  except  the  last  in  the 
series,  were  coded  in  one  of  three  ways:  R for  a general  response,  Ri  for  a response  directed  to  the  initiator  message, 
or  Rr  for  a response  to  a respondent  message  within  the  threaded  discussion.  The  last  message  in  the  threaded 
discussion  was  coded  similarly  to  the  response  messages:  C for  the  last  message,  Ci  for  a closing  message  that 
referred  to  the  initiator  message,  and  Cr  for  a closing  message  that  referred  to  a respondent  message.  Hence,  the 
coding  of  message  types  within  threaded  discussions  shows  the  connecting  patterns  of  the  response  messages. 

Coding  Message  Content 

Two  researcher  assumptions  about  the  online  communication  guided  this  comparative  inquiry  and  defined 
the  message  content  codes  created  and  used.  Message  content  codes  used  to  identify  specific  references  within  the 
body  of  posted  messages  were  developed  based  upon  the  following  assumptions.  First,  the  predominant  content  of 
instructor-generated  messages  was  about  online  assignments.  The  instructor  used  CEO  to  clarify  or  redirect  the 
assignment  or  offer  suggestions  for  action.  Despite  training  for  both  cohorts  in  the  use  of  the  university's  electronic 
communication  system  (CEO),  the  instructor  assisted  students  in  overcoming  uncertainty  or  confusion  about  using 
CEO  during  the  early  days  of  both  cohorts.  Thus,  content  codes  were  developed  to  flag  content  statements  related  to 
assignments,  suggestions  for  action,  and  technical  support.  Second,  cohort  use  of  online  communication  to  support 
student  learning  about  leadership  was  different  in  the  two  cohorts.  Message  content  codes  were  developed  to 
identify  references  to  leadership,  self-disclosure  statements  (e.g.,  statements  that  begin  with  the  words  I agree,  I 
disagree,  I believe,  I think),  and  professional  responsibilities  or  experiences. 

Table  L Coding  Key 

Message  Type 

Code  Description 

I Initiated  message 

R Response  (in  general) 

Ri  Response  to  initiator  message 

Rr  Response  to  respondent  message 

C Close  of  threaded  discussion  (last  message) 

Ci  Closed  as  response  to  initiator  message 

Cr Closed  as  response  to  respondent  message 

Message  Content 

Code  Description 

a Reference  to  assignment  (clarification,  redirection,  reminder,  wrap-up) 

n Suggestion  for  action 

r Reference  to  professional  responsibilities  and/or  experiences 

s Self  disclosure:  I agree,  I believe,  I disagree,  I feel,  I found,  I think 

t Reference  to  technical  support  and/or  trouble 

L Reference  to  leadership  theory  and/or  practice 

A matrix  for  analysis  of  online  messages  was  also  developed.  Dates  of  initiated  messages  and  closing 
responses  in  threaded  discussions  were  recorded  in  the  matrix.  Permission  to  analyze  online  messages  was  obtained 
from  all  participants  in  both  cohorts  prior  to  beginning  analysis.  Thus,  the  analysis  matrix  identifies  the  type  and 
content  of  each  message  generated  by  all  cohort  participants  so  that  comparisons  of  student  engagement  were 
possible.  An  ordered  triple  lists  the  number  of  initiated,  response,  and  closing  messages  made  by  each  individual 
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within  each  group  of  ten  messages.  A composite  totaling  all  messages  posted  by  each  cohort  member  within  the 
examined  conference  was  included  on  the  last  page  of  the  analysis  matrix  for  each  cohort. 

Additional  Message  Content  Analysis 

As  our  analysis  began  to  reflect  marked  differences  in  the  type  and  content  of  messages,  we  realized  that 
the  use  of  electronic  communication  for  instruction  was  not  as  similar  between  the  two  cohorts  as  we  first  thought. 
Needing  clarification  about  the  differences  between  the  DL  and  ECUS  syllabi,  we  asked  the  instructor  if  she  had 
changed  her  online  instructional  strategies.  We  learned  that  she  redesigned  the  online  assignments  in  the  ECUS 
cohort  to  encourage  reflection  and  discussion  in  a cohort-wide  online  conferencing  format.  In  the  DL  cohort, 
students  were  paired  and  thus  shared  their  reflections  and  discussions  in  private  messages,  which  were  not  available 
for  review  by  fellow  students  in  the  cohort  or  for  analysis  by  us.  Therefore,  we  realized  that  our  coding  of  only  the 
posted  messages  in  the  leadership  domain  of  the  DL  cohort  failed  to  present  a fair  comparison.  With  this  revelation, 
we  analyzed  student  interactions  within  the  five  DL  cluster  group  subconferences.  Since  cluster  groups  were  created 
within  geographic  regions  of  the  state  as  a way  for  students  to  support  one  another  in  face-to-face  settings,  we 
believed  that  a cluster  group  would  provide  opportunities  for  shared  learning  similar  to  those  in  the  onsite  cohort. 

Comparative  Findings:  Some  Surprises 

A cursory  scan  of  online  exchanges  within  the  leadership  domain  of  the  two  conferences  indicated  that  the 
two  cohorts  engaged  in  markedly  different  types  of  virtual  communication  during  the  beginning  six  months  of  each 
program.  Between  June  and  December  1999,  only  57  messages  (also  called  files)  were  posted  in  the  leadership 
domain  of  the  DL  cohort.  Thirty-three  of  the  57  (58%)  messages  were  instructor-generated.  Conversely,  although 
the  ECUS  cohort  met  together  as  a group  almost  weekly  from  January  to  July  2000,  the  cohort  generated  a total  of 
159  messages  in  the  leadership  conference.  Of  those  159  entries,  only  20  (13%)  were  posted  by  the  instructor.  An 
important  reason  for  these  variations  is  due  to  the  differences  between  the  CEO  conference  structures  for  the  two 
cohorts.  Most  of  the  DL  student  exchanges  for  the  leadership  domain  were  housed  in  cluster  group  subconferences. 
Nonetheless,  the  data  record  strongly  suggests  that  the  online  interaction  within  the  leadership  domain  of  the  DL 
cohort  was  less  interactive  when  compared  to  the  online  communication  among  members  of  the  ECUS  cohort. 

Content  of  Instructor  Messages 

Data  support  our  assumption  that  the  predominant  content  within  instructor-generated  messages  was  about 
online  assignments.  Of  the  33  DL  messages  posted  by  the  instructor,  25  (78%)  provided  clarification,  redirection,  or 
reminders  about  assignments  and  22  (67%)  suggested  action  to  be  taken  by  students.  Among  the  20  ECUS 
messages  initiated  by  the  instructor,  13  (65%)  referenced  an  assignment  and  12  (60%)  suggested  student  action.  As 
the  statistics  show  in  Table  2 and  Table  3,  the  instructor  used  CEO  in  both  cohorts  not  only  as  an  instructional  tool, 
but  also  as  a messaging  system. 


Table  2.  Instructor  Use  of  Online  Messages  for  Explaining  Assignments 


Cohort  Name 

Number  of  instructor- 
posted 
messages 

References  to  providing 
clarification,  redirection,  or 
reminders  about  work 

Percentage 

DL 

33 

25 

78% 

ECUS 

20 

13 

65% 

Table  3.  Instructor  Use  of  Online  Messages  to  Suggest  Action 

Cohort  Name 

Number  of  instructor- 

References  to  suggestions  for 

Percentage 

posted 

action  by  students 

messages 

DL 

33 

22 

67% 

ECUS 

20 

12 

60% 

ECUS  syllabus 

A review  of  the  ECUS  syllabus  indicated  that  the  instructor  explicitly  described  the  purposes  for  integrating 
online  activities  into  the  onsite  cohort  curriculum. 
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Traditionally,  this  domain  has  been  facilitated  in  a face-to-face  environment  with  some  e-conferencing  to 
supplement  instruction.  As  part  of  an  effort  to  increase  accessibility  to  material  away  from  the  classroom 
and  to  leverage  the  advantages  of  face-to-face  contact  in  the  classroom  time  available,  [the  ALPS  faculty] 
is  converting  more  of  the  material  into  a computer-supported  format.  Thirty  percent  of  the  course  material 
will  be  offered  on-line,  providing  opportunity  for  small  group  discussions  and  individual  reflection  between 
face-to-face  meetings  . . . learning  is  both  an  individual  and  interactive  process  that  involves  identity 
development  as  well  as  interpersonal  interaction  . . . assignments  [were  organized]  to  provide  extensive 
reading  and  study  on  one  hand  and  intensive  collaboration  on  the  other.  (Napier,  2000,  p.  1) 

The  first  e-conferencing  assignment  was  presented  verbally  to  the  ECUS  cohort  during  the  second  cohort  session 
(January  31,  2000)  and  clarified  during  the  third  meeting  (February  7,  2000).  However,  based  upon  a review  of 
observational  field  notes  and  a search  within  the  ECUS  cohort  conference,  written  directions  for  the  first  online 
assignment  were  not  distributed  during  a cohort  meeting  or  via  an  online  message.  Further,  an  interesting 
contradiction  in  our  comparative  findings  emerged  from  analysis  of  the  ECUS  student  comments  on  the  recent 
questionnaire. 

ECUS  student  reflections  about  the  online  assignment. 

The  student  responses  to  a survey  question  about  the  online  assignments  in  the  leadership  domain  provided 
interesting  contrasts  in  understanding.  Nine  of  the  17  (53%)  recalled  that  the  task  was  to  read  assigned  articles  and 
books  and  write  reactions  and  reflections  generated  by  the  readings.  Five  students  either  reported  that  they  could  not 
remember  specifics  about  the  assignments  or  they  left  the  answer  space  blank.  Another  student  wrote, 

I'm  not  sure  that  I really  understood  why  the  assignment  was  assigned.  I thought  it  was  more  of  an  inner 
reflection.  I did  not  feel  comfortable  expressing  my  thoughts  or  feelings  with  the  group.  I had  not  built  any 
trust. 

While  time  often  diminishes  recollection  of  details,  the  fact  that  6 of  the  17  (35%)  students  in  the  onsite 
cohort  could  not  recall  the  assignment  raises  a concern  about  assignment  clarity.  Students  may  have  asked  the 
instructor  individual  questions  about  the  online  activity  during  cohort  meetings.  However,  clarifications  about  this 
particular  assignment  do  not  appear  to  have  been  shared  with  the  entire  cohort  either  in  class  or  online.  The  lack  of 
written  instructions  for  the  first  online  activity  is  markedly  out  of  character  with  the  number  of  other  assignment 
messages  posted  by  the  instructor. 

An  Unexpected  Finding:  Need  for  Technical  Assistance 

During  analysis  of  instructor  messages  in  both  cohort  conferences,  we  noticed  an  interesting  finding  that 
emerged  concerning  the  need  for  technical  assistance.  Students  enrolled  in  both  the  DL  and  ECUS  cohorts  received 
training  in  the  use  of  the  university's  electronic  communication  system  prior  to  beginning  online  activities. 
Members  of  the  DL  cohort  received  intense,  hands-on  opportunities  to  practice  using  CEO  in  the  university's 
computer  laboratories  during  the  cohort’s  eight-day  program  orientation.  Online  training  for  the  ECUS  cohort 
included  a visual-display  presentation  by  a guest  speaker  during  the  cohort’s  second  meeting  and  one-on-one 
assistance  provided  by  both  the  instructor  and  peers  during  two  of  the  subsequent  class  meetings  in  a middle-school 
media  center.  Despite  training  in  the  use  of  the  communication  system,  data  indicate  that  students  in  both  cohorts 
reported  technical  difficulties  and  received  guidance  from  the  instructor  throughout  the  entire  timeframe  examined. 

References  to  technical  support  or  trouble  appeared  in  27  of  the  57  (47%)  of  DL  cohort  messages.  Eleven 
of  the  message  types  were  non-responded  messages  generated  by  the  cohort  instructor.  Six  message  types  were 
threaded  discussions  initiated  by  different  students  with  only  the  instructor  responding.  Only  two  of  the  threaded 
discussions  concerning  technology  in  the  DL  conference  were  initiated  by  students  and  responded  to  by  peers. 

References  to  technical  support  or  trouble  appeared  in  14  of  the  159  (9%)  of  the  ECUS  cohort  messages. 
Three  of  the  instructor-generated  non-responded  messages  link  technical  support  to  another  assignment.  Eight 
student  references  to  uncertainty  about  CEO  appeared  within  both  non-responded  messages  and  threaded 
discussions  among  students  during  the  first  two  weeks  of  the  assigned  online  activity.  The  last  reference  to  technical 
trouble  made  by  a student  was  found  in  a response  to  an  instructor's  message  about  completing  an  online 
assignment.  In  this  instance,  the  student  wrote  that  he  was  unable  to  participate  in  one  of  the  online  activities 
because  the  system  was  off-line  when  he  tried  to  complete  his  work. 


47 


58 


A comparison  of  the  number  of  references  made  by  students  about  having  difficulty  using  the  online 
communication  system  or  to  the  instructor  providing  technical  assistance  leads  to  interesting  questions.  Statistics 
are  provided  in  Table  4. 


Table  4.  Student  Need  for  Technical  Support 


Cohort  Name 

Total  number  of  messages 

References  to  technical 
support  or  trouble 

Percentage 

DL 

57 

27 

47% 

ECUS 

159 

14 

9% 

We  became  intrigued:  Does  the  fact  that  half  the  messages  posted  in  the  early  weeks  of  the  DL  cohort 
suggest  that  the  students  were  more  dependent  or  needy  than  the  students  in  the  onsite  cohort?  Or  does  the  small 
percentage  of  references  to  using  CEO  in  the  ECUS  cohort  mean  that  technical  problems  were  handled  in  class?  It 
may  be  tempting  simply  to  assume  from  the  statistics  that  the  DL  cohort  experienced  and  reported  greater  difficulty 
using  the  online  communication  system  than  the  ECUS  cohort  did.  However,  ECUS  student  responses  to  a recent 
questionnaire  about  using  CEO  suggest  that  learning  the  online  communication  system  created  problems  for 
students  in  the  onsite  cohort  as  well. 

ECUS  student  reflections  about  using  CEO. 

Eleven  of  the  17  (65%)  respondents  stated  that  they  did  not  encounter  major  difficulties  using  CEO.  The 
range  of  challenges  for  five  other  respondents  included  simply  not  being  able  to  log-on  because  of  an  incorrect 
spelling  of  user  name  to  needing  additional  training  from  the  CEO  administrator  or  a spouse.  However,  one  student 
reported  significant  difficulty  and  did  not  engage  in  online  activities  until  two  weeks  after  the  assignment  had  been 
made.  Once  she  learned  to  use  the  electronic  communication  system,  she  participated  as  actively  as  the  majority  of 
her  peers. 

I almost  died  learning  CEO.  It  took  several  weeks  to  feel  comfortable  understanding  and  using  the  system. 

I first  had  to  get  hooked  up  to  an  Internet  provider,  then  to  a search  engine  to  be  able  to  use  on  my  laptop. 

Then  I had  to  load  CEO  First  Class  . . . [and]  learn  how  to  navigate  around  in  the  system.  The  class  names 

were  even  foreign.  I worked  and  worked  until  I became  comfortable  using  the  system. 

Studnt  Learning  through  Online  Activities 

Our  quick  scan  of  the  data  record  (i.e.,  the  number  and  types  of  messages  in  each  leadership  conference) 
suggested  to  us  that  participants  in  the  ECUS  cohort  engaged  in  rich,  dialogue-like  exchanges  within  the  leadership 
domain  conference,  which  students  in  the  DL  cohort  did  not.  DL  students  may  have  engaged  in  very  similar 
discussions  with  another  student  in  the  cohort,  but  their  exchanges  occurred  within  private  mailbox  messages  or 
subconferences.  Thus,  students  in  the  DL  cohort  missed  opportunities  to  share  their  learning  and  understanding  with 
the  group  at-large. 

References  to  leadership. 

A review  of  the  statistics  shows  how  very  different  the  contents  of  the  messages  are  in  each  of  the  cohort 
conferences.  The  word  "leadership"  was  used  in  58  of  the  139  (42%)  ECUS  student  messages,  but  it  appeared  only 
3 times  in  the  24  (13%)  DL  student  messages.  This  finding  both  amazed  and  puzzled  us.  We  purposefully  selected 
the  leadership  domain  conference  in  each  cohort  as  the  course  for  our  comparative  analysis,  assuming  that  both 
cohorts  followed  a similar  structure.  We  learned  from  the  instructor  that  she  observed  the  same  phenomenon  in  the 
DL  cohort  leadership  conference  that  we  did.  Thus,  she  purposefully  made  a dramatic  change  in  the  format  of  the 
online  curriculum  for  the  ECUS  cohort  in  order  to  create  a structure  for  sharing  learning  cohort-wide. 

References  to  self-disclosure  statements. 

We  measured  the  vitality  of  online  communication  by  evidence  of  the  number  of  times  that  students  made 
self-disclosure  statements  in  which  they  used  the  following  phrases:  I agree , I believe,  I disagree,  I feel,  I found,  I 
support,  or  I think . Self-disclosure  statements  appear  in  1 1 0 of  the  1 39  (79%)  ECUS  student  messages  and  14  of  24 
(58%)  DL  student  messages.  An  interesting  observation  is  that  only  six  instructor  messages  in  the  ECUS  cohort 
contained  self-disclosure  statements.  Conversely,  the  instructor  used  self-disclosure  statements  in  25  of  her  33 
(76%)  messages  in  the  DL  cohort. 
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References  to  professional  practice. 

Additionally,  data  indicate  that  students  in  the  ECUS  cohort  attempted  to  connect  what  they  were  learning 
in  the  licensure  program  to  their  professional  responsibilities  or  experiences.  Fifty-one  of  the  139  (37%)  student 
messages  include  reflections  in  which  professional  responsibilities  are  mentioned.  None  of  the  student  messages  in 
the  DL  conference  make  a reference  to  professional  responsibilities  or  experiences.  A summary  of  findings  is 
presented  in  Table  5. 


Table  5.  Student  Learning  through  Online  Activities 


Cohort 

Number  of  student- 

References  to 

Made  self- 

References to 

posted  messages 

leadership 

disclosure  statement 

professional 

experiences 

DL 

24 

3 

14 

0 

ECUS 

139 

58 

110 

51 

DL  cluster  groups. 

The  disparity  between  the  leadership  messages  compelled  us  to  look  elsewhere  in  the  DL  cohort.  Five 
cluster  groups  were  formed  during  the  orientation  session  of  the  DL  cohort  in  June  1999  with  the  intent  to  build 
local  communities  of  support  for  the  geographically  dispersed  cohort  members.  An  interesting  finding  is  the 
incredibly  wide  range  of  student  online  interactions  within  the  five  cluster  group  subconferences  during  the  entire 
yearlong  program.  The  least  active  group  posted  only  35  messages  while  the  most  active  one,  known  as  WestSide, 
exchanged  344  messages.  The  other  three  cluster  groups  generated  a total  of  83,  67,  and  64  files  each.  The  range  in 
quantity  of  exchanges  among  the  five  cluster  groups  poses  interesting  questions  about  group  dynamics,  which  were 
not  addressed  in  this  inquiry.  What  is  significant  about  this  finding  is  that  the  WestSide  cluster  group  was  also  the 
most  social  cluster  group  in  the  DL  cohort.  The  intensity  and  volume  of  peer-to-peer  interactions  within  the  ECUS 
cohort  and  the  WestSide  cluster  group  suggest  that  peer-to-peer  support  and  encouragement  developed  because  of 
opportunities  for  face-to-face  interaction.  Analysis  to  date  of  the  interaction  within  the  DL  cluster  group 
subconferences  shows  a greater  number  of  student  exchanges  than  in  the  leadership  conference.  Although  the 
messages  within  the  DL  leadership  conference  were  somewhat  stagnant,  the  richness  of  cohort  activities  became 
apparent  within  cluster  group  subconferences.  References  to  leadership  practices,  professional  growth,  and  the 
challenges  of  balancing  the  licensure  program  requirements  with  professional  responsibilities  were  mentioned  in  the 
subconferences. 

ECUS  student  reflections  about  online  learning. 

Statistical  analysis  provides  only  one  perspective  of  student  reaction  to  the  required  e-conferencing 
activities.  Thus,  we  returned  to  the  ECUS  questionnaire  for  more  clarity  of  findings.  Only  7 of  the  1 7(42%)  student 
responses  to  the  recent  ECUS  questionnaire  provided  positive  assessment  of  the  online  assignment.  One  of  those 
students  wrote  a particularly  enthusiastic  evaluation. 

I personally  love  this  type  of  assignment.  I believe  that  [it]  is  appropriate  for  the  graduate  level  and  gets 
people  out  of  the  "regurgitation  of  information"  mode.  As  I observe  others  in  the  cohort,  I detect  that  many 
are  not  dedicated  to  this  method  of  learning  . . . This  assignment,  however,  [prodded  us]  to  think  deeply  and 
investigate,  evaluate  and  speculate. 

An  interesting  observation  about  this  student  in  cohort  meetings  is  that  over  the  past  10  months  he  rarely 
participated  in  face-to-face  discussions.  Like  other  more  reticent  students,  he  seems  to  prefer  online  sharing. 

I appreciate  the  convenience  of  online  discussions  ...  the  discussion  topics  and  the  venue  allows  people  to 
be  more  introspective  and  share  personally.  I think  there  is  a huge  potential  for  online  components. 

Other  students  who  enjoyed  doing  the  online  leadership  activity  wrote: 

Definitely  a new  and  fresh  approach!  I like  [online  activities]  because  you  can  sit  at  home  and  have  time  to 
reflect  and  write  your  thoughts. 
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I enjoyed  the  technological  side  of  this  assignment.  Using  a computer  and  the  Internet  to  discuss  class 
topics  is  very  fascinating.  Doing  this  kind  of  assignment,  however,  made  me  nervous  because  I had  not 
explored  myself  as  a true  leader. 

However,  almost  half  of  the  ECUS  cohort  did  not  share  the  same  reaction  to  being  assigned  online  dialogue 
activities.  Five  students  reported  feeling  overwhelmed,  perplexed,  worried,  and  even  threatened  about  writing  self- 
disclosing  reflections. 

I struggled  with  this  assignment  because  I truly  had  not  seen  myself  as  a leader.  I was  apprehensive  to  put 
those  words  online  for  all  to  read. 

I felt  threatened  because  I was  asked  to  expose  my  inner  thoughts  to  20  people  that  I did  not  know. 

Approximately  one-third  of  the  respondents  reported  that  they  did  not  enjoy  the  online  conferencing  for  other 
reasons.  It  was  frustrating  for  some  when  peers  did  not  respond  to  the  messages  they  initiated  in  the  hopes  of 
beginning  threaded  discussions.  Others  voiced  a preference  for  face-to-face  discussions  in  which  there  is  increased 
potential  for  peer  interaction. 

[Not  receiving  responses  to  posted  messages]  affected  me  by  making  me  feel  that  what  I had  to  say  was  not 
important.  I also  began  to  feel  like  "why  should  I even  do  this  if  it  is  not  even  read  or  responded  to  by 
anyone." 

[Using  CEO  for  assignments]  is  a nice  up-to-date  way  of  trading  ideas  and  work.  I would  like  to  see  it  used 
more,  but  nothing  beats  face-to-face  communication  and  learning. 

Comparison  of  student  engagement 

A line-by-line  analysis  of  type  and  content  of  student  messages  in  each  of  the  leadership  conferences 
suggests  that  many  students  never  fully  participated.  Analysis  of  student  responses  in  the  DL  leadership  conference 
matrix  indicates  that  four  students  never  posted  a message  there  during  the  entire  program.  Analysis  of  the  ECUS 
matrix  shows  that  many  students  were  hesitant  during  the  early  weeks  to  share  self-disclosures  with  their  peers.  One 
student  did  not  engage  in  the  process  until  three  weeks  after  the  assignment  was  made.  Among  the  four  messages  he 
posted  in  the  conference,  only  one  contains  a self-disclosure  statement.  Another  student  did  not  post  any  messages 
until  almost  five  weeks  after  the  assignment  was  made.  Of  the  seven  messages  authored  by  this  student,  only  four 
contain  self-disclosure  statements. 

Conclusions  and  Possible  Questions  for  Discussion 

Comparative  analyses  of  the  online  messages  in  both  conferences  strongly  suggest  that  neither  cohort  spent 
time  establishing  ground  rules  for  online  behavior.  What  emerged  during  the  process  of  online  communication 
worked  for  some  students,  but  not  for  others.  Thus,  the  use  of  the  online  conferencing  system  as  an  instructional 
tool  differed  significantly  between  the  two  principal  licensing  cohorts. 

The  DL  cohort  used  the  CEO  system  more  as  a communication  tool  for  discussion  of  logistical  issues 
related  to  their  learning  and  assignments  rather  than  as  a platform  for  academic  discourse.  Only  one  of  the  five 
cluster  groups  within  the  DL  cohort  sustained  rich  online  communication  throughout  the  entire  program.  Perhaps 
one  reason  for  the  online  connectedness  of  this  DL  cluster  group  was  the  fact  that  the  students  initiated  and  sustained 
regular  face-to-face  meetings  to  support  one  another. 

The  onsite  ECUS  cohort  used  the  CEO  system  as  an  instructional  tool  because  the  purpose  for  online 
activities  was  clearly  delineated  in  the  leadership  course  syllabus.  Confusion  about  the  first  assignment  arose  for 
two  reasons:  (a)  the  assignment  was  not  written  and  (b)  some  students  had  difficulties  learning  to  use  CEO  during 
the  early  weeks  of  the  program.  Nonetheless,  the  majority  of  the  ECUS  students  posted  personal  reflections  and 
engaged  in  rich,  peer-to-peer  online  discourse  about  their  assigned  readings  and  related  their  learning  to  professional 
responsibilities.  In  some  ways  their  online  dialogue  rivaled  discussions  observed  during  weekly  cohort  meetings. 

Because  our  study  did  not  compare  truly  similar  online  learning  environments  or  include  large  numbers  of 
students,  our  findings  are  limited.  Results  of  our  analysis  of  the  content  of  student  messages  provide  important 
information  to  assist  the  ALPS  faculty  in  the  design  of  future  distance-learning  programs  and  online  assignments 
within  onsite  cohorts.  Student  responses  to  the  questionnaire  about  use  of  CEO  in  the  professional  preparation 
program  contain  some  surprising  revelations  that  can  further  assist  in  program  modification.  Adult  learning  styles 
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vary  tremendously  and  thus  accommodations  need  to  be  made  so  students  have  ready  access  to  technical  assistance 
in  the  use  of  information  technology  during  the  early  weeks  of  a new  program.  Further,  cohort  norms  need  to  be 
established  early  regarding  participation  in  online  activities. 

Findings  in  our  comparative  study  support  the  literature  base  and  empirical  evidence  from  other  studies 
about  online  communication.  A greater  value  is  that  our  findings  generated  questions  for  discussion  about  the 
effectiveness  of  online  learning  activities. 

Question  One:  Student  Engagement  in  Online  Activities 

Data  from  our  comparative  study  show  that  not  all  students  in  the  two  cohorts  readily  embraced  the  idea  of 
actively  participating  in  asynchronous  online  dialogue.  While  some  students  in  the  ECUS  cohort  reported  enjoying 
online  activities,  their  peers  revealed  reticence  for  sharing  personal  views  and  reflections  in  the  public  domain  of  an 
online  conference.  Several  students  reported  having  technical  difficulties  during  the  early  weeks  of  both  programs. 
Further,  three  ECUS  students  posted  three  or  fewer  messages  among  a total  of  139  student  files,  and  four  of  the 
students  in  the  DL  cohort  never  posted  a message  within  the  leadership  domain  conference.  A stated  purpose  for 
assigning  online  activities  is  to  enhance  collaborative  learning.  Therefore,  if  student  participation  is  important  for 
learning,  then  what  type  of  monitoring  does  an  instructor  need  to  use  to  assess  student  online  engagement? 

Question  Two:  Online  Conferencing  and  Student  Learning  Styles 

In  face-to-face  classroom  situations,  some  students  are  very  willing  to  share  their  thoughts  and  ideas  openly 
as  fuel  for  discussion  and  debate  among  peers.  Other  students  prefer  to  listen  and  observe  classroom  exchanges  and 
reflect  about  what  was  said.  An  instructor  can  notice  facial  expressions  and  body  language  to  discern  if  quiet 
students  are  engaged  in  the  classroom  activity  and  even  elicit  responses  through  direct  questioning.  In  a virtual 
environment,  however,  an  instructor  cannot  always  discern  if  "quiet"  students  are  actively  participating.  The  fact 
that  a message  has  been  opened  does  not  mean  that  it  was  read.  Therefore,  what  strategies  can  an  instructor  use  to 
monitor  active  student  participation  in  online  activities? 

While  our  comparative  study  of  message  content  was  intended  to  assist  the  ALPS  faculty  in  the  design  of 
the  next  distance-learning  cohort  and  the  integration  of  more  online  activities  in  onsite  cohorts,  our  findings  suggest 
questions  for  discussion  with  larger  audiences.  We  begin  that  larger  dialogue  with  this  presentation. 
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How  To  Develop  Cognitive  Flexibility 
In  A WWW  Course 

Ana  Amelia  Amorim  Carvalho 
University  of  Minho 


Abstract 

Cognitive  flexibility  is  indispensable  for  applying  knowledge  to  new  situations.  The  development  of  this 
ability  depends  on  certain  conditions  such  as  the  attainment  of  a deep  comprehension  of  the  subject  matter  and  the 
exposure  to  different  knowledge  representation.  This  paper  focuses  on  these  conditions  and  describes  a study 
designed  to  foster  the  development  of  cognitive  flexibility  on  a World  Wide  Web  course. 

Cognitive  flexibility 

Cognitive  flexibility  is  the  ability  to  change  one’s  perspective,  to  categorize  data  and  stimuli  according  to 
different  properties,  to  find  new  connections  among  the  elements  of  a whole  and  to  interpret  the  same  reality  in 
different  ways.  It  is  also  the  ability  to  recombine  elements  of  a representation,  or  to  reorder  the  importance  of 
elements  in  different  contexts  (Spiro  et  al.,  1987).  Flexibility  in  thinking  allows  subjects  to  move  from  one  category 
to  another  and  to  modify  their  point  of  view  (Guilford,  1967). 

Spiro  & Jehng  (1990:  165)  state:  "by  cognitive  flexibility,  we  mean  the  ability  to  spontaneously  restructure 
one's  knowledge,  in  many  ways,  in  adaptive  response  to  radically  changing  situational  demands".  Moreover,  "this  is 
a function  of  the  way  knowledge  is  represented  (e.g.,  along  multiple  rather  single  conceptual  dimensions)  and  the 
processes  that  operate  on  those  mental  representations  (e.g.,  processes  of  schema  assembly  rather  than  intact  schema 
retrieval)".  According  to  these  authors  cognitive  flexibility  depends  on  the  way  knowledge  is  represented.  Bearing 
this  in  mind,  next  session  focuses  on  knowledge  representation,  particularly  on  complex  knowledge  representation. 

Knowledge  representation 

The  representation  of  complex  knowledge,  according  to  several  authors,  has  to  avoid  compartmentalization, 
simplification,  and  a single  dimension  of  analysis  (Barthes,  1970;  Morin,  1990;  Spiro  et  al.,  1991).  Multiple 
dimensions  of  analysis  are  necessary  for  developing  cognitive  flexibility  that  depends  of  having  a diversified 
repertoire  of  ways  of  thinking  about  a conceptual  topic. 

"Interpreter  un  texte,  ce  n'  est  pas  lui  donner  un  sens  (plus  ou  moins  fonde,  plus  ou  moins  libre);  c'  est  au  contraire 
apprecier  de  quel  pluriel  il  est  fait"  (Barthes,  1 970:  1 1). 

Cognitive  Flexibility  Theory  (CFT)  proposes  principles  that  help  to  develop  cognitive  flexibility  (Spiro  & Jehng, 
1990),  such  as  "knowledge  deconstruction"  and  "thematic  criss-crossing".  At  this  point  it  is  important  to  stress  that 
this  theory  is  case-based.  A case  represents  specific  knowledge  tied  to  a context.  It  may  be  a chapter  of  a book,  a few 
frames  of  a film,  an  event.  Cases  may  have  different  shapes  and  sizes,  covering  large  or  small  time  slices  (Spiro  & 
Jehng,  1990;  Kolodner,  1993;  Kolodner  & Leake,  1996).  Each  case  has  to  be  divided  in  small  parts,  called  mini- 
cases. Each  mini-case  is  analyzed  according  to  multiple  dimensions  or  multiple  perspectives:  the  themes. 

Themes  are  synonymous  of  principles  or  rules  and  they  help  to  understand  the  complex  knowledge.  Each 
perspective  or  theme  gives  a new  insight  into  the  mini-case  comprehension. 

According  to  CFT  there  are  two  important  paths  to  develop  cognitive  flexibility:  "knowledge 
deconstruction"  and  "thematic  criss-crossing".  In  the  next  section  we  describe  the  two  paths  illustrating  them  with  an 
example:  the  web  document  "Cousin  Basilio:  multiple  thematic  Criss  Crossings"  (available  at  the  following  URL: 
www.iep.uminho.pt/primobasilio).  ",  for  literary  studies,  we  select  a 19th  century  novel,  "Cousin  Basilio"  written  by 
E<?a  de  Queiros. 
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Two  complementary  ways  of  knowledge  representation  to  promote  cognitive  flexibility 

Knowledge  deconstruction 

The  notion  of  knowledge  deconstruction  is  shared  by  R.  Barthes,  J.  Derrida  and  R.  Spiro  and  it  stresses  the 
importance  of  multiple  perspectives  to  deeply  understand  a subject  matter.  According  to  CFT  the  process  of 
knowledge  deconstruction  implies  the  selection  of  themes  and  cases.  The  case  is  divided  in  small  parts,  the  mini- 
cases. For  each  mini-case  it  is  necessary  to  identify  the  relevant  themes.  Then,  for  each  applied  theme  a "Thematic 
Commentary"  has  to  be  written,  explaining  how  each  theme  applies  to  the  particular  mini-case.  This  text  (thematic 
commentary)  should  state  clear  ideas  in  a simple  language  and  it  should  be  short  to  be  easily  read  in  a computer 
screen  (see  figure  1). 


Figure  1 - Mini-case  and  Thematic  Commentary  of  "Cousin  Basilio" 

In  the  web  course  that  we  developed  "Cousin  Basilio:  multiple  thematic  Criss  Crossings",  we  identified 
nine  themes  to  approach  the  novel.  This  novel  has  sixteen  chapters  that  we  grouped  to  constitute  the  cases  (table  1). 
Each  case  was  divided  in  small  parts,  the  mini-cases.  Then,  we  identified  the  relevant  themes  to  each  mini-case  and 
we  wrote  the  thematic  commentaries  (see  table  1). 

For  example,  four  themes  apply  to  the  first  mini-case  of  Case  I (see  figure  1).  The  Thematic  Commentaries 
related  to  each  of  these  themes,  give  different  and  complementary  insights  to  the  text.  On  figure  1,  one  can  see,  on 
the  right  side,  a mini-case  and  below  it  the  text  of  the  first  Thematic  Commentary. 


Chapters 

Cases 

Mini- 

Cases 

Thematic 

Commentaries 

I -III 

I 

8 

37 

IV- V 

II 

7 

37 

VI  - VII 

III 

6 

35 

VIII -XIII 

IV 

9 

36 

XIV  - XVI 

V 

4 

15 

Total 

- 

34 

160 

Table  1 - Components  of  the  process  of  deconstruction  of  “Cousin  Basilio" 

As  "Cousin  Basilio"  action  is  carried  out  in  the  19th  century,  for  each  mini-case  we  included  information 
about  the  epoch.  This  information  focuses  on  clothes,  furniture,  transportation,  writers,  composers,  operas  or  novels 
mentioned  in  the  mini-case.  This  information  includes  not  just  text  but  also  pictures  and  video  clips. 
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Flexibility  in  applying  knowledge  depends  on  mini-cases  being  deconstructed  (through  themes  and  thematic 
commentaries;  the  user  attains  a deep  understanding  of  that  mini-case)  and  it  also  depends  on  rearranged  structural 
sequences  from  different  points  of  view  or  perspectives  (thematic  criss-crossings).  Knowledge  that  have  to  be  used 
in  many  ways  has  to  be  learned,  represented  and  tried  out  in  many  ways. 

Thematic  criss-crossing 

The  process  of  thematic  criss-crossing  is  inspired  in  Ludwig  Wittgenstein'  book  "Philosophical 
Investigations".  According  to  Spiro  & Jehng  (1990),  Cognitive  Flexibility  Theory  generalizes  Wittgenstein’s 
metaphor  of  the  criss-crossed  landscape.  The  authors  explained  that  "by  criss-crossing  topical/conceptual 
landscapes,  highly  interconnected,  web-like  knowledge  structures  are  built  that  permit  greater  flexibility  in  the  ways 
that  knowledge  can  potentially  be  assembled  for  use  in  comprehension  or  problem  solving”  (Spiro  & Jehng,  1990: 
170). 

After  the  selection  of  a theme  (or  a combination  of  themes),  thematic  criss-crossing  guides  the  user  through 
a sequence  of  mini-cases  (of  different  cases)  and  thematic  commentaries  to  which  the  selected  theme  applies.  For 
example,  figure  2 exemplifies  the  process  of  thematic  criss-crossing.  In  that  example  it  is  used  the  first  traversal: 
"Denouncement  of  decadence".  The  sequence  presented  to  the  user  does  not  have  to  respect  the  cases  sequence 
(figure  2),  but  it  has  to  be  meaningful  and  give  a deeper  and  multifaceted  understanding  of  the  theme. 


Figure  2 - Thematic  Criss-Crossing  (Denoucement  of  decadence) 


"The  same  content  material  is  covered  in  different  ways,  at  different  times,  in  order  to  demonstrate  the  potential 
flexibility  of  use  inherent  in  that  content"  (Spiro  et  al.,  1988:  379). 


Figure  3 - Thematic  Criss-crossing  (mini-case) 
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Both  processes  are  complementary  in  the  kind  of  learning  promoted,  and  they  have  to  be  explored  alternately.  As 
users  explore  the  mini-cases  and  the  thematic  criss-crossings,  they  will  be  familiar  with  some  mini-cases  that  they 
can  read  for  different  purposes.  Bearing  this  in  mind  we  decided  to  use  the  bold  style  to  give  evidence  to  several 
expressions,  sentences  or  words  that  are  relevant  to  the  thematic  criss-crossing  (see  figure  3).  In  a glance  the  user 
identifies  the  mini-case,  then  he/she  will  focus  on  the  bold  text. 

The  importance  of  cognitive  flexibility  to  knowledge  transfer  to  new  situations 

Transferring  knowledge  to  new  situations  is  a very  demanding  task.  This  level  of  transfer  is  considered  to 
be  the  most  difficult  and  is  referred  to  as  "far  transfer"  (Gick  & Holyoak,  1987).  It  is  necessary  to  master  a subject 
and  have  the  ability  to  restructure  the  knowledge  to  fit  or  solve  the  new  situation,  e.g.,  it  is  necessary  to  have 
cognitive  flexibility.  The  mentioned  authors  consider  also  two  other  levels  of  transfer:  "self  transfer"  or  knowledge 
reproduction  and  "near  transfer"  where  there  is  a similarity  between  the  new  situation  and  others  analyzed. 

Structure  of  the  World  Wide  Web  Course 

We  developed  a literary  studies  course  in  the  web  to  support  the  study  of  a novel,  "Cousin  Basilio",  that 
describes  the  social  life  in  Lisbon  in  the  19th  century  (http://www.iep.uminho.pt/primobasilio),  as  we  mentioned 
above.  During  the  design  phase  we  took  in  attention  the  utilization  of  icons  and  colors  that  helped  to  recreate  the 
19th  century  ambience  in  the  web  document  "Cousin  Basilio : multiple  thematic  criss-crossings".  For  example,  the 
Thematic  Commentary  background  is  an  ancient  official  Portuguese  stationery  and  the  mini-cases  background 
represents  an  old  (yellow)  page,  inviting  the  user  to  pursue  due  to  its  slightly  rolled  page  on  the  right  comer  (figure 
1). 


Menul 

Mainl 

Menu2 

Main2 

Footer 

Figure  4 - Web  site  areas 

The  web  pages  are  structured  according  to  three  main  areas  (figure  4).  From  top  to  bottom,  we  have  the 
Menu  1,  that  is  the  Main  Menu.  If  we  select  an  item  on  this  menu  the  information  will  be  available  on  the  Main  1 
(figure  1).  Then,  on  the  area  below,  we  have  the  Menu  2.  This  menu  is  dynamic  because  the  items  available  on  this 
menu  depend  on  the  options  selected  on  the  Menu  1.  The  item  selected  on  the  Menu  2 will  be  available  on  the  Main 
2 (see  figure  1).  The  last  area  is  called  footer,  and  besides  copyright  information  and  e-mail  address,  there  is  also  the 
possibility  of  the  user  to  write  his/her  personal  notes,  clicking  on  the  pen  (see  figure  5). 

The  menu  1 offers  four  options:  Cases  (knowledge  deconstruction);  Thematic  Criss-Crossing;  Search;  and 
Table  of  Contents.  On  the  menu  2 we  have  access  to  " thematic  commentaries ",  information  about  the  Context  of  the 
19th  century  and  about  the  text  (mini-case)  of  the  novel,  and  a general  description  about  the  nine  Themes  selected  for 
approaching  the  novel.  Some  mini-cases  have  pictures  that  help  to  understand  Lisbon  scenarios  or  some  ancient 
transportation  or  even  some  ancient  furniture  that  helps  to  recreate  and  understand  that  century.  Other  mini-cases 
have  a video  that  presents  information  about  the  novel.  Finally,  at  the  end  of  Menu  2,  we  have  References , listing  all 
authors  mentioned  in  "thematic  commentaries"  or  in  the  general  description  of  Themes. 

Instructions  are  provided  to  the  user  each  time  he/she  selects  an  item  (path)  on  the  Menu  1.  These 
instructions  in  blue  simulate  well-designed  handwriting  (figures  5 and  6). 
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Figure  5 - Instructions  to  the  user 


* 


Figure  6 - Search  (cases  and  themes) 

The  home  page  has  an  image  of  an  ancient  book.  It  gives  access  to  the  Help  page.  On  this  page,  one  may 
obtain  information  about  the  approach  used  to  the  novel  and  about  the  options  available  on  both  menus.  If  the  coins 
are  pressed,  one  has  access  to  the  login  page  (figure  7).  User  name  and  password  will  be  required.  This  document  is 
access  free,  however,  the  password  is  needed  to  save  one's  own  notes  (during  a month  from  last  access).  Feel  free  to 
look.  These  web  pages  are  discrete  and  simultaneously  appealing.  Most  of  the  web  courses  available  have  a high 
text  density  , which  is  not  motivating  for  the  user!  Why  aren't  we  exploring  the  web  multimedia  potentialities? 
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Figure  7.-  User  name  and  Password  (access  free) 

The  study 

In  this  study  we  evaluate  how  far  the  course  structure  contributes  to  the  development  of  cognitive 
flexibility.  We  use  two  groups  and  two  different  documents.  One  of  the  documents  gives  access  to  all  CFT  facilities 
(named  CFT)  and  the  other  one  does  not  have  access  to  Thematic  Commentaries  (named  NTC  - No  Thematic 
Commentaries),  but  only  to  the  applied  themes  to  each  mini-case. 

We  developed  several  instruments  to  collect  data  such  as  knowledge  tests  (for  measuring  far,  near  and  self- 
transfer  questions);  Questionnaires  of  Opinion  to  collect  users'  opinions  about  the  web  document  design  and 
structure  and  their  orientation  on  it,  path  preference  and  involvement  experienced,  a Questionnaire  on  computer 
literacy  and  about  user's  motivation  to  participate  in  this  study,  and  a final  report  about  this  study  and  their  opinion 
about  web  courses  for  further  learning. 

As  a pre-requisite  for  this  study,  subjects  should  be  Portuguese  Literature  undergraduate  students  and  they 
have  had  to  read  the  novel  "Cousin  Basilio".  Subjects  received  three  packages  along  the  course,  containing 
instructions  for  the  sessions  on  the  web;  knowledge  tests:  pre-test,  intermediate  test  and  post-test  and  Questionnaires 
of  Opinion  about  the  WWW  course,  and  a Questionnaire  on  Computer  Literacy. 

When  they  finished  the  first  package  (Pre-test,  Questionnaire  on  Computer  Literacy),  they  contact  the 
researcher  for  a session  in  the  lab.  This  session  intends  to  help  subjects  to  feel  comfortable  to  use  and  explore  the 
web  document.  At  the  end  of  this  session,  students  took  the  second  package.  As  soon  as  they  finished,  the  last 
package  was  sent  to  each  one.  Finally  when  they  sent  this  one  they  received  the  questions  for  writing  the  final 
report. 

Sample  characterization 

Twenty  eight  3rd  year  undergraduate  students  enrolled  in  Portuguese  Literature  participated  in  this  study, 
three  males  and  twenty-five  females,  ranging  from  nineteen  to  twenty-seven  years  old.  According  to  the  information 
collected  by  the  Questionnaire  on  Computer  Literacy,  we  realized  that  most  of  the  subjects  (79%)  never  explored  an 
interactive  environment  such  as  CD-I  or  CD-R.  Only  12  subjects  had  explored  the  Internet.  They  mentioned 
different  kinds  of  motivation  for  participating  in  this  study,  but  the  most  referred  ones  were:  to  participate  in  a 
distance  learning  web  course,  to  participate  in  this  study  due  to  their  interest  in  literary  work,  or  due  to  their  interest 
in  E<?a  de  Queiros'  literary  work. 

Results  and  discussion 

We  used  non-parametric  tests,  specifically  Mann- Whitney  U test.  The  knowledge  tests  measure  three  kinds 
of  transfer:  self  transfer  or  reproduction,  near  transfer  and  far  transfer  (Gick  & Holyoak,  1987).  The  kinds  of  transfer 
that  measure  cognitive  flexibility  are  near  and  far  transfer.  This  last  one  is  the  most  difficult  and  demanding  kind  of 
transfer. 
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Pre-test 

CFT 

NTC 

Z corrected 
for  ties 

Statistical 

significance 

Total 

15.19 

13.58 

-.511 

p-,6091  NS 

Reproduction 

14.59 

14.38 

-.071 

p=.9434  NS 

Near  transfer 

15.44 

13.25 

-.699 

p=,4846  NS 

Far  transfer 

14.48 

14.04 

-.260 

p=.7951  NS 

Table  2 - Pre-test  statistical  analysis  (Mann- Whitney  U test) 

Pre-test  results  show  that  there  is  no  statistically  significant  difference  (p=.6091),  the  groups  are  similar 
before  being  submitted  to  treatment  (table  2). 


Intermediate 

test 

CFT 

NTC 

Z corrected 
for  ties 

Statistical 

significance 

Total 

18.03 

9.79 

-2.62 

p=.0087  S 

Reproduction 

14.75 

14.16 

-.186 

p-  8531  NS 

Near  transfer 

17.87 

10 

-2.51 

p-0121  S 

Far  transfer 

18.12 

9.66 

-2.71 

p=.0066  S 

Table  3 - Intermediate  test  statistical  analysis  (Mann-Whitney  U test) 

During  this  study,  students  have  done  an  intermediate  test.  Results  pointed  out  to  statistically  significant 
difference  achieved  in  the  test  (p<.05)  as  in  near  and  far  transfer  questions.  There  is  no  statistically  significant 
difference  in  the  reproduction  questions,  perhaps  because  the  knowledge  to  be  applied  to  these  questions  was  the 
same  on  both  documents. 


Post-test 

CFT 

NTC 

Z corrected 
for  ties 

Statistical 

significance 

Total 

18.12 

9.67 

-2.69 

p=0071  S 

Reproduction 

14.50 

14.50 

0 

p=l  NS 

Near  transfer 

17,66 

10.29 

-2.35 

p=.0189  S 

Far  transfer 

17.12 

11.00 

-2.02 

p=. 043  S 

Table  4 - Post-test  statistical  analysis  (Mann-Whitney  U test) 

Post  test  results  show  that  there  is  statistically  significant  difference  (p=.043)  that  gives  evidence  to  the 
development  of  cognitive  flexibility  on  the  group  of  subjects  exploring  the  CFT  web  document.  This  result  shows 
the  importance  of  thematic  commentaries  on  knowledge  transfer  to  new  situations  (table  4).  CFT  group  has  better 
results  in  near  and  far  transfer  questions.  As  on  the  previous  test,  both  groups  achieved  similar  results  in  these 
question,  p=l. 

Most  of  subjects  (66%)  considered  the  session  in  the  lab  indispensable  and  34%  of  subjects  considered  that 
the  information  available  on  "help"  was  clear  enough  to  explore  the  document.  In  what  concerns  users'  opinion 
about  the  web  document,  58%  considered  it  "accessible  to  use"  and  42%  considered  it  "easy"  to  learn  to  use.  On  the 
first  session,  most  of  them  (58%)  felt  oriented  in  the  web  document  and  37%  felt  disoriented;  on  the  following 
sessions  all  feel  oriented,  excepted  one  subject  who  felt  disoriented. 

Most  of  them  (84%)  considered  the  reading  proposed  to  the  novel  "interesting"  and  16%  considered  it 
"acceptable".  They  mentioned  that  they  (95%)  felt  actively  involved  in  this  learning  process.  Their  preferred  path  is 
the  deconstruction  process  (Cases), 

Subjects'  opinion  about  web  courses  for  further  learning  is  a positive  one  (74%),  however  some  would 
prefer  a combination  of  face-to-face  meetings  and  distance  learning. 

Conclusion 

The  knowledge  representation  used  in  this  study  to  promote  cognitive  flexibility  gives  emphasis  to 
knowledge  deconstruction  and  to  thematic  criss-crossing.  This  complementary  approach  to  complex  knowledge 
representation  used  by  Cognitive  Flexibility  Theory  led  us  to  study  what  is  the  importance  of  "thematic 
commentaries"  (this  is  the  focus  of  the  deconstruction  process)  in  learning  and  in  cognitive  flexibility.  Results  give 
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evidence  to  the  deconstruction  process  to  develop  cognitive  flexibility,  i.e.,  to  the  importance  of  "thematic 
commentaries"  to  knowledge  transfer  to  new  situations. 

Although  subjects'  computer  literacy  was  low  they  felt  it  was  easy  to  use  the  web  document  and  to  navigate 
in.  They  liked  its  design  and  structure. 

They  are  receptive  to  participate  in  web  courses  for  further  learning,  but  some  would  prefer  a combination 
of  face-to-face  meetings  and  distance  learning. 


References 

Barthes,  R.  (1970).  S/Z . Paris:  Editions  du  Seuil. 

Carvalho,  A.  A. A (1999).  Os  Hipermedia  em  Contexto  Educativo.  Aplicagao  e validagao  da  Teoria  da 
Flexibilidade  Cognitiva . Braga:  CEEP,  Universidade  do  Minho. 

Gick,  M.L.  & Holyoak,  K.J.  (1987).  The  cognitive  basis  of  knowledge  transfer.  In  S.  M.  Cormier  e J.  D. 
Hagman  (eds),  Transfer  of  Learning : Contemporary  Research  and  Applications.  New  York,  NY : Academic  Press, 

9-46. 

Guilford,  J.P.  (1967).  The  nature  of  human  intelligence . New  York:  McGraw-Hill. 

Kolodner,  J.  L.  (1993).  Case-Based  Reasoning.  San  Francisco:  Morgan  Kaufmann. 

Kolodner,  J.  L.  & Leake,  D.  B (1996).  A Tutorial  Introduction  to  a Case-based  Reasoning.  In  D.  B.  Leake 
(ed)  Case-based  Reasoning . Experiences,  lessons&  future  directions . American  Association  for  Artificial 
Intelligence,  31-66. 

Morin,  E.  (1990).  Introdugao  ao  Pensamento  Complexo.  Lisboa:  Instituto  Piaget. 

Spiro,  R.,  Feltovich,  P.J.,  Jacobson,  M.  & Coulson,  R.  (1991)  Cognitive  Flexibility,  Constructivism,  and 
Hypertext:  random  access  instruction  for  advanced  knowledge  acquisition  in  ill-structured  domains.  Educational 
Technology , XXXI,  5,  24-33. 

Spiro,  R.  & Jehng,  J.-C.  (1990).  Cognitive  Flexibility  and  Hypertext:  theory  and  technology  for  the 
nonlinear  and  multidimensional  traversal  of  complex  subject  matter.  In  Don  Nix  & R.  Spiro  (eds.),  Cognition, 
Education,  and  Multimedia:  Exploring  Ideas  in  High  Technology.  Hillsdale,  NJ.  Lawrence  Erlbaum  Associates, 
163-205. 

Spiro,  R.,  Vispoel,  W.p.,  Schmitz,  J.G.,  Samarapungavan,  A.  & Boerger,  A.E.  (1987).  Knowledge 
Acquisition  for  Application:  Cognitive  Flexibility  and  Transfer  in  Complex  Content  Domains.  In  B.  C.  Britton  & 
S.  M.  Glynn  (eds.),  Executive  Control  in  Processes  in  Reading.  New  Jersey:  Lawrence  Erlbaum  Associates,  1 77- 
199. 

Wittgenstein,  L.  (1987).  Tratado  Logico-Filosofico.  Investigates  Filosoficas.  Lisboa:  Funda^ao  Calouste 
Gulbenkian. 


o 

ERIC 


60 


69 


Learning  Science  Concepts  at  a Distance  in  Preservice  Teacher  Education: 

Results  of  a Pilot  Study 

Christina  Wotell  Chamitski 
College  of  Misericordia 

Francis  A.  Harvey 
Drexel  University 


Abstract 

This  paper  will  report  the  findings  of  the  first  phase  of  a multi-phased  research  project  that  is  aimed  at 
examining  preservice  teachers  ’ development  of  science  concepts  when  learning  (a)  takes  place  at  a distance,  and  (b) 
is  facilitated  by  videoconferencing  technology . This  study  will  build  on  the  work  that  was  done  by  Lansdown, 
(Lansdown,  Blackwood,  and  Brandwien,  1971)  and  validated  at  Boston  College  (Trainor,  1978)  and  at  Harvard 
University  (Harvey,  1980)  that  demonstrated  the  validity,  effectiveness  and  evaluative  power  of  an 
investigation/colloquium  approach  to  students'  development  of  science  concepts  in  a face-to-face  learning 
environment.  The  pilot  study  investigated  the  use  of  the  investigation/colloquium  approach  (a  series  of  student- 
directed  explorations  with  hands-on  materials  followed  by  colloquium  discussions  facilitated  by  an  adult  leader)  by 
ten  students  at  two  sites  connected  by  videoconferencing  technology . Results  of  the  pilot  study,  analyzed  using 
standard  qualitative  analysis  procedures,  indicated  that  Vygotskian  group  socio-cultural  learning  (particularly  the 
development  of  science  concepts)  could  take  place  effectively  when  the  groups  are  in  separate  locations  connected 
by  videoconferencing  technology. 

Introduction  and  Need  for  the  Study 

This  paper  reports  the  findings  of  a pilot  study  which  was  the  first  phase  in  a project  designed  to  investigate 
and  validate  techniques  for  using  videoconferencing  technology  to  promote  the  development  of  preservice  teachers* 
science  concepts.  The  ultimate  goal  of  this  three-phased  study  is  the  development  of  a curriculum  matrix  (content, 
material,  and  procedures)  that  will  facilitate  science  concept  learning  at  a distance.  The  study  will  be  used  to 
examine  the  efficacy  of  using  videoconferencing  technologies  to  improve  science  concepts  in  preservice  teachers  by 
(a)  supporting  innovative  course  configurations  that  may  increase  preservice  teachers’  access  to  pedagogically  sound 
science  instruction,  (b)  extending  professional  collaboration  between  preservice  and  inservice  teachers;  and  (c) 
promoting  quality  continuing  education  that  is  accessible  and  affordable. 

The  results  of  the  Third  International  Mathematics  and  Science  Survey  (IEA,  1996;  NSB,  1998)  raised 
serious  concerns  about  science  education  in  the  United  States.  The  data  collected  from  the  TIMSS  demonstrated  that 
the  U.S.  science  curriculum  lacks  focus  and  depth  (NSB,  1998;  Schmidt  & Wang,  1999;  Valverde  & Schmidt, 
1998).  Schmidt  and  Wang  (1999)  stated: 

One  of  the  most  salient  and  powerful  policy  implications  from  the  TIMSS  is  the  essential  role  of  the 
curriculum  in  the  teaching  and  learning  of  math  and  science.  The  TIMSS  achievement  results  suggest  a disturbing 
pattern:  the  relative  standing  of  U.  S.  students  compared  with  other  countries  declines  from  fourth  to  twelfth  grade. 
The  U.S.  science  curriculum  over  these  same  grade  levels  has  been  characterized  as  highly  repetitive,  lacking 
coherence,  and  not  focusing  on  rigorous  content  as  defined  internationally,  (online) 

Schmidt  and  Wang  suggested  that  the  failure  of  the  U.  S.  students  in  science  may  well  be  the  consequence 
of  “...the  mile  wide  inch  deep  curriculum,  the  largest  textbooks  in  the  world,  and  a splintered  vision  of  what 
children  should  learn  about  science.”  (online)  They  further  noted  that  U.S.  textbooks  “...play  a major  role  in  the 
splintering  or  fragmenting  that  occurs  in  U.  S.  science  education.”  (online)  They  argued  that  the  problem  with 
textbooks  cannot  be  underemphasized  since  teachers  in  elementary  or  middle  school  who  do  not  have  strong  science 
backgrounds  view  and  teach  science  directly  from  the  textbooks. 

Research  literature  suggests  that  most  U.S.  elementary  and  middle  school  teachers  lack  the  content 
knowledge  and  confidence  needed  to  teach  science  (Berg,  Huinker,  & Neuman,  1993;  Rice  & Corboy,  1995)  . 
According  to  Rutherford  and  Ahlgren  (1990): 

Few  elementary  school  teachers  have  even  a rudimentary  education  in  science  ...  Unfortunately,  such 
deficiencies  have  long  been  tolerated  by  the  institutions  that  prepare  teachers,  the  public  bodies  that  license  them, 
and  the  schools  that  hire  them.... 
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A revised  curriculum  is  needed  in  both  the  content  and  the  delivery  of  preservice  science  courses  if  we  are 
to  prepare  teachers  who  have  a firm  understanding  of  the  basics  of  science  (AAAS,  1994;  Berg  et  al.,  1993;  Harris, 

1 993;  Tolman  & Campbell,  1991). 

Measuring  learner  achievement  has  been  a major  focus  of  studies  in  interactive  video-based  learning 
(Moore,  Thompson,  Quigley,  Clark  and  Goff,  1990;  Payne,  1997).  A vast  body  of  research  literature  generally 
acknowledges  that  there  is  no  significant  difference  in  learner  achievement  between  students  who  participate  in 
instruction  at  a distance  and  students  who  participate  in  traditional  settings.  Yet,  the  validity  of  this  body  of  research 
has  been  called  into  question  (Cohen,  Ebeling,  & Kulik,  1981;  Moore  et  al.,  1990;  Phipps  & Merisotis,  1999; 
Russell,  1996;  Schlosser  & Anderson,  1994).  Phipps  and  Merisotis,  1999  stated  that  there  is  rising  evidence  that 
much  of  the  existing  research  that  has  examined  student  achievement  in  distance  learning  is  weak  and  largely 
anecdotal. 

Most  research  on  distance  learning  has  treated  learner  achievement  as  an  all-inclusive  term  that  does  not 
distinguish  between  higher  and  lower  level  thinking  skills.  Vygotsky  (1986)  identified  concept  development  as  the 
ability  to  (a)  form  linkages  between  concepts  without  reference  to  concrete  or  sensory  impressions,  (b)  verbally 
define  the  concept  in  abstract  terms,  and  (c)  apply  the  concept  to  new  situations  which  must  be  considered  in 
abstract  terms.  Clearly,  learner  achievement  that  is  a measure  of  concept  development  as  defined  by  Vygotsky  is  far 
different  than  learner  achievement  that  is  a measure  of  lower  level  cognitive  skills  such  as  the  student’s  ability  to 
identify,  order,  or  compare  lists  of  items. 

This  paper  will  report  the  findings  of  the  first  phase  of  a three-phase  study  that  was  conducted  in  March- 
April  2000.  The  participants  included  undergraduate  students  from  College  Misericordia’s  Teacher  Education 
Program  who  were  enrolled  in  a science  methods  class.  A unit  on  waves  was  taught  using  the 
investigation/colloquium  model.  Half  of  the  students  participated  from  the  videoconferencing  facilities  at  College 
Misericordia  (Dallas,  PA)  while  the  other  half  participated  from  King’s  College  (Wilkes  Barre,  PA). 

Preliminary  findings  suggest  that  preservice  teachers  can  learn  science  concepts  at  a distance  when 
connected  by  videoconferencing  technology  using  the  investigation/colloquium  method  of  instruction.  Other 
preliminary  data  offer  insights  into  procedural  changes  for  the  second  phase  of  the  study  which  will  take  place  in 
September  of  2000. 

Method 

A considerable  body  of  literature  (e.g.,  Lenning  & Ebbers,  1999;  Moller,  1998;  Mukhopadhyay,  1997; 
Palloff  & Pratt;  Stone  & Sulino,  1997)  has  suggested  that  a key  element  in  successful  distance  learning  is  the 
formation  of  a synthetic  class  relationship.  A synthetic  class  relationship  is  one  in  which  the  students  come  together 
as  one  cohesive  community  of  learners.  In  a successful  distance  synthetic  class  relationship,  students  feel  and  act  as 
part  of  one  group  whose  members  just  happen  to  be  situated  in  various  geographical  locations.  The  verbal  and 
nonverbal  socio-cultural  interactions  evident  in  a synthetic  class  relationship  are  consonant  with  Vygotsky's 
emphasis  on  the  socio-cultural  aspect  of  concept  development.  This  pilot  study  was  conducted  in  the  late  spring  of 
2000.  The  study  was  designed  to  extract  information  from  the  socio-cultural  elements  in  the  learning  environment 
by  focusing  on  the  patterns  of  interaction  between  and  among  participants  during  the  colloquium  segments  of  each 
videoconferencing  session. 

The  research  took  place  at  a small,  private,  four-year  college  in  Northeastern  Pennsylvania  and  included  a 
baseline  assessment  of  the  participants  and  four  one-hour  videoconferencing  sessions.  The  study  participants 
included  10  volunteer  preservice  education  majors  who  had  completed  their  core  science  requirements  but  had  not 
yet  fulfilled  their  science  methods  requirement.  Five  of  the  students  participated  from  the  college  videoconferencing 
facilities  and  five  students  participated  from  a videoconferencing  facility  approximately  10  miles  from  the  campus. 
Students  were  designated  as  being  same-site  or  other-site,  which  was  determined  by  the  physical  presence  or 
absence  of  the  instructor. 

The  instructor  was  a physics  professor  who  has  a background  in  science  education,  and  regularly  co- 
teaches the  science  methods  courses  with  a teacher  education  faculty  member.  During  the  study  the  instructor  was 
physically  present  at  each  site  for  two  sessions.  Additionally,  an  activity  aide  was  present  at  the  other  site  (i.e., 
remote)  who  simply  handed  out  materials  and  did  not  interact  with  the  students  during  the  sessions. 

The  study  consisted  of  a baseline  assessment  and  investigation/colloquium-based  lessons  that  were 
facilitated  by  videoconferencing  technologies.  The  baseline  assessment  consisted  of  a short  one-on-one  interview  of 
the  participants  to  establish  their  baseline  knowledge  of  waves.  Participants  were  asked  an  open  ended  question 
(i.e.,  “What  can  you  tell  me  about  waves?”),  and  a follow  up  question  (i.e.,  Is  there  anything  more  you  can  tell  me?) 
The  participants  exhibited  wave  concept  understanding  below  an  upper  elementary  level. 
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The  experimental  sessions  were  comprised  of  four  one-hour  investigation/colloquium  sessions  that  were 
delivered  using  videoconferencing  technologies.  The  first  three  videoconferencing  sessions  began  with  the 
instructor  introducing  manipulate  materials  that  would  allow  the  participants  to  physically  explore  wave 
phenomena.  The  materials  used  were:  (1)  slinkies;  (2)  string  telephones;  (3)  oven  racks  suspended  by  strings.  After 
the  instructor  introduced  the  materials,  the  students  were  given  ten  minutes  during  which  they  engaged  in  hands-on 
investigations  using  the  materials.  At  the  end  of  the  ten  minutes  of  investigation,  the  instructor  called  both  groups 
together  to  participate  in  a fifteen-minute  colloquium.  The  colloquia  were  student-directed,  with  the  instructor 
merely  acting  as  a facilitor  to  the  discussion.  The  purpose  of  the  colloquia  was  to  have  the  participants  discuss  their 
observations  and  attempt  to  verbally  make  sense  of  their  investigations.  Among  other  things  the  colloquium 
participants  asked  questions,  gained  insights  from  each  other,  and  came  up  with  new  ideas  about  how  further 
investigation  may  help  them  find  answers  to  their  questions.  Following  the  colloquium  the  participants  engaged  in 
another  ten  minutes  of  hands-on  investigation,  which  was  again  followed  by  a fifteen-minute  colloquium.  During 
the  final  week,  the  session  structure  remained  the  same  (i.e.,  ten  minutes  of  investigation  followed  by  fifteen  minutes 
of  colloquium)  however,  the  participants  watched  video  segments  that  demonstrated  wave  concepts  instead  of 
working  with  manipulate  materials. 

All  session  were  audiotaped  and  videotaped.  Written  transcriptions  were  made  from  the  tapes.  Colloquium 
statements  were  analyzed  using  Harvey’s  (1980)  scoring  scheme.  Verbal  interactivity  was  analyzed  by  numerical 
occurrences  relative  to  the  directionality  of  the  statement.  Each  statement  was  coded  as:  (1)  a response  to  a 
statement  from  either  the  same  site  or  the  other  site;  (2)  a statement  directed  to  either  the  same  site  or  the  other  site. 

Results  and  Analysis 

Informal  analysis  of  the  participants’  statements  from  the  colloquia  associated  with  the  hands-on 
investigations  demonstrated  that  students  at  both  the  same-site  as  the  facilitator  and  at  the  remote  site  developed 
higher-level  concepts.  The  participants’  statements  were  analyzed  using  Harvey’s  (1980)  scheme,  which  is  based 
on  Vygotsky’s  levels  of  concept  development.  The  statement  analysis  indicated  an  increase  in  students’  higher 
level  thinking  statements  (i.e.,  pre-concept)  and  a decrease  in  lower  level  thinking  statements  (i.e.,  complex)  over 
the  three  sessions.  In  addition,  the  student’s  responses  became  more  concise  and  focused  as  the  sessions  progressed 
and  the  total  number  of  student  statements  decreased  with  each  session. 

Instructor  (facilitator)  statements  also  decreased  over  time  which  corresponded  with  a decrease  in  student 
statements  directed  to  the  instructor.  Instructor  statements  accounted  for  29%  of  the  total  statements  during  the  first 
session  and  8%  of  the  statements  in  the  final  session.  Total  student  statements  (same-site  and  other-site)  directed  to 
the  instructor  (facilitator)  decreased  from  51  during  the  first  session,  to  26  during  the  second  session,  to  one  during 
the  final  session. 

Extensive  verbal  interaction  occurred  both  within  sites  and  between  sites,  and  the  interactions  between  sites 
increased  over  the  three  colloquia.  Figures  1 through  3 present  the  number  and  types  of  responses  made  by  subjects 
during  the  three  sessions  to  statements  from  the  facilitator,  from  subjects  at  the  other  location,  and  from  subjects  at 
the  same  location.  Figures  4 through  6 present  the  number  and  types  of  statements  directed  by  subjects  to  the 
facilitator,  to  subjects  at  the  other  location,  and  to  subjects  at  the  same  location. 
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Discussion 

The  main  emphases  of  science  reform  appear  to  be  consistent  with  Vygotsky's  so cio- cultural  theory  of 
learning.  The  initial  findings  of  this  study  suggest  that  the  unique  characteristics  of  videoconferencing  technology 
make  it  a promising  medium  for  delivery  of  effective  Vygotskian-based  learning  in  teacher  education.  All  indicators 
point  to  videoconferencing  at  both  the  studio  and  desktop  level  becoming  more  generally  accessible,  and  therefore, 
more  likely  to  be  used  in  educational  settings  involving  K-12  students. 

Recently,  the  validity  and  reliability  of  much  of  the  existing  research  on  the  use  of  videoconferencing  has  been 
called  into  question.  This  has  left  a paucity  of  reliable  research  that  explains  the  phenomena  related  to  teaching  and 
learning  via  videoconferencing.  Many  critical  questions  remain  unanswered.  Further,  the  development  of  higher 
level  thinking  skills  and  concept  formation  is  at  the  heart  of  the  science  reform  recommendations;  yet  the  existing 
body  of  literature  fails  to  make  a distinction  between  higher  and  lower  level  thinking  skills  as  they  relate  to  student 
achievement. 

Videoconferencing  and  other  distance  technologies  alter  the  classroom  dynamics  sufficiently  to  warrant  a 
cohesive  research  effort  that  examines  the  mediating  factors  that  affect  student  concept  development  when  learning 
takes  place  at  a distance.  The  results  of  this  study  suggest  that  the  investigation/colloquium  approach,  which  has 
proven  successful  in  the  traditional  classroom,  may  be  useful  in  videoconferencing  learning.  The  final  two  phases  of 
this  research  project  will  further  test  and  refine  the  preliminary  findings  of  this  study. 
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Abstract 

Recent  advancements  in  computer  and  Internet  technologies  enable  universities  to  implement  cost-effective 
Web-Based  Instruction  (WBI)  and  to  provide  open  learning  environments  24  hours  a day,  7 days  a week . While 
more  and  more  WBI  courses  are  continually  being  developed,  little  attention  is  being  paid  to  effective,  systemic,  and 
systematic  WBI  design  and  development.  Collis  (1996)  mentions  “ WWW-based  course  environments  are  rapidly 
appearing,  before  there  has  been  time  for  much  theoretical  development  with  respect  to  guidelines  for  their  design  ” 
(p.  26).  Although  there  are  some  general  guidance  systems  focused  on  WBI  design  and  development  process,  much 
of  the  knowledge  about  these  guidance  systems  is  tentative  and  lacking  in  details.  The  purpose  of  this  research, 
therefore,  is  to  improve  these  guidance  systems  and  to  provide  more  detailed  and  useful  guidelines  in  a special 
context. 

This  research  first  synthesized  several  general  WBI  design  and  development  guidance  systems.  Next,  the 
formative  research  methodology  was  used  to  improve  that  synthesis.  A case  was  chosen  for  this  study,  the  context 
being  the  teaching  of  music  fundamentals  at  the  pre-college  level.  Efforts  were  made  to  identify  which  guidelines 
were  or  were  not  useful  in  this  case  and  which  guidelines  might  be  beneficial  to  modify,  delete,  or  add  in  this 
context.  Both  on-site  and  online  interviews,  observations,  and  document  analyses  were  conducted  with  all 
developers  involved  in  this  WBI  project. 

As  a result  of  this  study,  the  synthesized  general  WBI  design  and  development  guidance  system  was  revised 
for  best  fit  with  this  case.  These  revisions  are  the  major  findings  of  this  research.  Discussion  of  each  guideline  and  a 
final  summary  of  the  results  were  also  made.  Recommendations  for  practitioners  and  future  research  directions 
were  also  discussed.  This  new  guidance  system  was  intended  not  only  to  guide  future  practitioners  in  this  field,  but 
also  to  provide  a theoretical  framework  for  future  research  and  theory  development. 

Background  of  the  study 

The  single  case  in  this  study  is  a project  called  “Music  Fundamentals  Online  (MFO).”  According  to  the 
MFO  project  director,  the  typical  undergraduate  core  in  music  schools  in  the  United  States  consists  of  a four-  or 
five-semester  sequence  of  parallel  courses  in  music  theory  and  in  musical  skills  (dictation,  sight  singing,  keyboard, 
etc.),  followed  by  one  to  two  years  of  music  history. 


1st  semester  [Fundamentals] 

2nd  semester 

Year  1 (Freshman) 

Theory  and  Literature  I (T 151), 
Musical  Skills  I 

Theory  and  Literature  II 
Musical  Skills  II 

Year  2 

Theory  and  Literature  III 
Musical  Skills  III 

Theory  and  Literature  IV 
Musical  Skills  IV 

Year  3 

Theory  and  Literature  V 
Musical  Skills  V 

Year  4 

History  I 

History  II 

Table  I:  Core  curriculums  at  a typical  music  school  (Isaacson,  1998) 


An  important  part  of  the  core  courses  is  prior  mastery  of  music  fundamentals.  The  term  “music 
fundamentals”  refers  to  knowledge  and  skills  associated  with  basic  music  literacy.  Typically  it  includes  the  ability  to 
read  treble  and  bass  clefs,  to  write  and  identify  scales,  intervals,  key  signatures,  and  chords,  and  to  know  basic 
elements  pertaining  to  music  notation,  rhythm,  and  meter.  It  also  usually  includes  basic  aural  skills,  such  as 
recognition  of  intervals  and  chord  types,  the  ability  to  write  down  simple  pitch  and  rhythm  patterns,  and  a limited 
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amount  of  sight  singing.  Although  it  is  important  to  have  basic  music  skills  before  entering  music  schools,  many 
students  entering  college  who  plan  to  have  music  as  their  major  lack  mastery  of  these  basic  skills.  Take  Indiana 
University  for  example.  The  MFO  project  director  pointed  out  that  over  half  of  the  students  beginning  the  core 
curriculum  lack  proficiency  in  this  area.  Other  institutions  report  figures  ranging  from  20-80%  with  50%  being 
typical  (Isaacson,  1998).  These  students,  therefore,  require  remediation. 

Removing  this  deficiency  at  the  college  level  is  very  expensive  for  both  students  and  music  schools- 
US$420-$1300  per  student  per  semester  as  of  1999  for  tuition  at  Indiana  University  (depending  on  the  residency 
status),  plus  textbooks  and  other  costs.  It  is  expensive  for  the  music  schools,  too,  because  schools  need  to  hire  extra 
instructors  and  allocate  educational  resources  (classrooms,  musical  instruments,  equipment,  and  so  on)  for  this 
course.  It  is,  therefore,  to  the  student’s  and  to  the  school’s  advantage  that  students  arrive  on  campus  with  a solid 
grounding  in  music  fundamentals.  As  a solution  to  this  problem,  Indiana  University  has  taken  the  challenge  and  is 
developing  an  innovative  WBI  remediation  course  (MFO)  in  the  hope  of  replacing  the  present  classroom-based 
music  fundamentals  course.  Students  who  register  in  MFO  will  devote  four  to  six  weeks  to  complete  the  whole 
course  online.  Upon  completion  of  the  prototype,  the  MFO  program  will  be  available  during  the  summer  before 
students  enter  college/music  schools. 

Guidance  Systems  of  WBI  Design  and  Development 

WBI  is  still  in  its  infancy,  and  there  is  little  research  about  WBI  design  and  development  guidance  systems. 
A number  of  general-purpose  guidance  systems  for  WBI  design  and  development,  though  general  in  nature  and  not 
usually  intended  for  complex  WBI  courses,  can  illustrate  both  the  trends  and  process  involved.  To  limit  the  scope  of 
the  study,  some  guidance  systems  that  focused  mainly  on  GUI  issues  or  product  issues  (e.g.,  Collis,  1997;  Boyle, 
1997;  Harasim,  Calvert,  & Groeneboer,  1997;  Hedberg,  1997;  Santi,  1997;  Dillon  & Zhu,  1997;  Horton,  2000)  were 
intentionally  omitted  in  this  literature  review  because  they  are  more  product-oriented.  Following  are  some  of  the 
general  WBI  design  and  development  guidance  systems. 

Berge’s  Guiding  Principles  in  WBI  design 

Berge  (1998)  first  defines  design  as  “planning  the  instructional  programme  events— building  the  blueprint  to 
guide  development  and  tryout”  (p.  38).  This  definition  is  similar  to  Wien  and  Gunter’s  “design”  stage  of  WBI  (Wien 
& Gunter,  1998). 

Berge  also  limits  the  scope  of  his  WBI  design  guiding  principles  and  excludes  some  of  the  guidelines  for 
on-line  moderating  and  teaching  (more  product-oriented).  According  to  Berge  (1998),  rules  such  as  “find  unifying 
threads,”  “present  conflicting  opinions,”  and  “a  facilitator  should  encourage  the  participants  to  introduce 
themselves,”  are  intentionally  left  out  in  his  guiding  principles  as  they  are  too  narrowly  focused  on  implementation 
and  delivery  of  on-line  teaching. 

Berge  (1998)  made  some  key  assumptions  before  presenting  his  guiding  principles  in  WBI  design.  These 
key  assumptions  include: 

• Learning  is  a lifelong  process  that  is  important  to  effective  participation  in  cultural  and  economic  life 
in  a democratic  society. 

• Learning  involves  the  development  of  a broad  range  of  skills,  knowledge,  and,  particularly,  attitudes 
that  can  be  and  should  be  fostered  in  both  formal  and  informal  learning  environments. 

• Learning  involves  a social  construction  of  knowledge.  (Berge,  1998,  p.  32) 

He  then  listed  eight  guiding  principles  for  WBI  design  and  categorized  them  into  three  groups  (Berge,  1998). 

• Pedagogical 

o Define/describe  and  list  the  purpose(s)  for  each  activity,  level  and  type  of  social  and  instructional 
interactivity,  and  feedback  that  is  desired. 

o Define  the  levels  of  teacher-control,  guided-teacher-control,  student  control  and  group-control  that 
are  desired  regarding  each  activity. 

o Density  of  content  should  be  inversely  related  to  the  amount  of  synchronous  communication 
within  the  Web-based  educational  learning  environment. 

• Technical/support 

o Recognize  that  while  on-line  environments  such  as  the  Web  permit  multiple-media,  currently  text 
and  graphics  are  the  easiest  to  use. 
o Use  the  principle  of  technological  minimalism. 

o Adequate  technical  support  and  training  for  both  student  and  instructor  is  essential. 
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• Social 

© An  important  goal  of  web-based  learning  is  the  creation  of  an  environment  of  co-operation  and 
trust  among  students  and  the  instructor. 

© In  general,  synchronous  communication  is  more  expensive  than  asynchronous.  Still,  both 
synchronous  and  asynchronous  modes  of  communication  are  important  web-based  tools  in  teaching 
and  learning  (Berge,  1998,  p.  33). 

First  of  all,  among  all  of  Berge’s  guiding  principles,  some  of  them  seem  to  be  rather  descriptive  and  lack 
detailed  instruction  or  action  to  tell  practitioners  what  to  do  and  how  to  do  it.  For  example,  the  guideline  “Density  of 
content  should  be  inversely  related  to  the  amount  of  synchronous  communication  within  the  Web-based  educational 
learning  environment”  does  not  include  further  instruction  about  “how  to  do  it.”  This  guideline  is  also  considered  to 
be  somewhat  product-oriented,  as  a more  process-oriented  guideline  would  be  “Analyze  the  density  of  content  to  be 
taught  within  the  Web-based  learning  environment.”  The  other  descriptive  guideline  is  “Recognize  that  while  on- 
line environments  such  as  the  Web  permit  multiple-media,  currently  text  and  graphics  are  the  easiest  to  use.”  Also, 
since  this  is  a general-purpose  WBI  design  guidance  system,  some  of  the  guiding  principles  tend  to  be  general  or 
imprecise.  For  example,  the  second  principle  in  the  pedagogical  group  can  also  be  an  ISD  principle  and  not 
specifically  a WBI  design  principle.  There  are  many  different  genres  of  WBI  courses.  Some  WBI  courses  are  more 
like  on-demand  CBI  (Clark,  1996)  or  CMI  delivered  via  the  Web.  Whether  all  WBI  courses  require  social  activities 
as  Berge  suggested  (in  his  assumption)  is  debatable.  It  will  be  beneficial  to  improve  these  guiding  principles  in  a 
specific  context  based  on  formative  research. 

Welsh’s  Event-Oriented  Design  (EOD)  Model  for  WBI 

As  Welsh  (1997)  pointed  out,  “those  designing  instruction  that  uses  the  Web  as  the  primary  means  of 
communication  between  class  participants  need  instructional  development  models  that  take  into  account  the  current 
and  future  capabilities  of  the  Web,  as  well  as  its  evolving  limitations”  (p.  159).  Welsh  first  mentioned  that  any 
instructional  design  model  for  WBI  must  meet  the  following  criteria: 

1.  It  must  be  systematic,  and  therefore  useful  as  a standard  online  course  development  methodology. 

2.  It  must  be  adaptable  to  different  educational  disciplines  and  to  differing  pedagogical  orientations. 

3.  It  must  be  technology  independent,  incorporating  technologies  in  wide  use  for  instruction,  as  well  as  new 

technologies  such  as  the  Web. 

4.  It  must  be  useful  in  traditional  contexts  so  faculty  can  recognize  the  benefits  of  the  design  approach  in 
instructional  contexts  other  than  WBI  (Welsh,  1997,  p.  160). 

According  to  Welsh  (1997),  the  EOD  model  involves  consideration  of  three  elements  that  draw  from  the 
fields  of  distance  education  and  instructional  design.  These  three  elements  are  “asynchronous  vs.  synchronous 
learning,  specification  of  performance  objectives  and  the  determination  of  instructional  strategies  for  meeting 
objectives,  and  specification  of  information  technologies  best  suited  to  meet  instructional  goals  in  distance  contexts” 

(p.  160). 

In  the  EOD  model,  first  a course  is  conceptualized  as  a series  of  individual  modules.  Each  module  is 
comprised  of  a series  of  instructional  events,  each  of  which  results  in  students  meeting  specific  performance 
objectives.  In  summary,  designing  for  WBI  using  the  EOD  model  involves  the  following  steps: 

1.  Specify  instructional  goals  and  performance  objectives  of  the  course  using  traditional  instructional 
design  methods. 

2.  Sequence  performance  objectives  and  chunk  them  into  a series  of  instructional  modules,  each  of  which 

results  in  students  meeting  objectives.  While  instructional  modules  need  not  be  equal  in  duration  or 

scope,  parallel  structuring  can  establish  a comfortable  rhythm  for  the  students  and  instructor. 

3.  Divide  each  module  into  a series  of  instructional  events 

4.  For  each  event,  specify  event  types:  full  synchronous,  limited  synchronous,  or  asynchronous. 

5.  For  each  event,  specify  appropriate  Web-based  technology  to  enable  the  event.  Care  should  be  taken  to 

choose  only  from  Web-based  technologies  available  to  the  instructor  and  all  students. 

6.  For  each  event,  develop  Web-based  content  where  needed  and  define  procedures  that  ensure  smooth 
completion  of  the  event. 

7.  Engage  in  formative  evaluation  and  pilot  testing  as  necessary  to  verify  that  each  event,  as  well  as  the 
course  as  a whole,  is  robust  pedagogically  and  procedurally  (Welsh,  1997,  p.  162-163). 

Although  Welsh  (1997)  does  not  define  what  he  means  by  “parallel  structuring,”  according  to  this  article 
(Welsh,  1997),  he  was  referring  to  the  sequencing  and  chunking  of  the  course  modules  (e.g.,  similar  sequencing 
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structure,  duration,  or  scope  for  each  instructional  module).  While  Welsh’s  EOD  model  tries  to  model  WBI  after 
traditional  classroom-based  instruction,  I agree  with  other  scholars  (Khan,  1997;  Relan  & Gillani,  1997)  that  WBI 
may  have  the  potential  to  be  more  flexible  and  powerful  (e.g.,  more  individualized  or  personalized  instruction)  than 
traditional  classroom-based  instruction.  Also,  although  ideally  it  would  be  great  to  have  a guidance  system  that  is 
adaptable  to  different  educational  disciplines  and  to  differing  pedagogical  orientations,  this  is  a goal  that  cannot  be 
easily  achieved.  To  be  more  adaptable  to  different  educational  disciplines  and  to  differing  pedagogical  orientations, 
a guidance  system  sometimes  tends  to  be  more  general  and  lack  specifics  and  details  for  a certain  context.  The  EOD 
model  can  also  benefit  from  formative  research  and  more  case  studies  for  further  improvement. 

Gibson  and  Herrera’s  Case  Study 

Gibson  and  Herrera  (1997)  did  a case  study  “How  to  go  from  classroom-based  to  online  delivery  in 
eighteen  months  or  less:  A case  study  in  online  program  development.”  They  described  how  a traditional 
undergraduate  classroom-based  course  was  redesigned  to  online  delivery  and  the  several  stages  of  design  and 
development.  This  study  provided  the  following  recommendations  (guidance)  for  the  WBI  design  and  development 
process: 

1 . Decide  upfront  if  your  goal  is  to  simply  put  some  courses  online  or  to  design  an  entire  online  program.  If 
the  former,  the  resources  needed  are  much  less.  If  you  are  not  sure  whether  your  faculty  or 
administration  or  even  your  technical  system  will  support  an  online  program,  start  by  developing  a few 
courses  and  offering  them  to  current  students 

2.  Use  an  existing  course  of  studies,  hopefully  one  that  you  have  had  much  success  with  so  that  you  are  not 
doing  curriculum  development  and  learning  how  to  teach  online  at  the  same  time. 

3.  Identify  enthusiastic  faculty  champions  right  away.  Faculty  support  is  the  most  important  element;  you 
cannot  succeed  without  it.  We  recommend  that  you  choose  only  full-time  faculty  at  the  outset;  bringing 
in  outsiders  will  forever  diminish  the  status  of  the  program  to  “continuing  education.” 

4.  Allocate  the  financial  resources  to  pay  your  faculty  developers.  Online  development  is  very  time 
consuming,  and  although  you  are  providing  new,  marketable  skills  to  the  faculty  participants,  there  is  an 
opportunity  cost  to  them. 

5.  Treat  your  developers  as  a team;  hold  frequent  meetings.  They  need  to  share  ideas  and  help  each  other 
stay  focused.  There  is  much  frustration  during  the  learning  curve.  Reinforce  their  work  and  recognize 
their  accomplishments  at  every  opportunity. 

6.  Make  sure  that  technical  support  is  readily  available  to  faculty  and  students.  This  includes  having  the 
right  software  and  hardware  provided  to  faculty  and,  most  importantly,  having  technical  people  ready  to 
help  the  faculty  whenever  they  may  need  it.  Build  in  this  same  level  of  technical  support  for  students 
when  the  classes  begin. 

7.  Do  whatever  you  can  to  assure  that  your  university  has  an  adequate  online  library.  Students  taking  online 
classes  are  doing  so  for  the  freedom  from  logistical  boundaries.  The  online  library  services,  for  example, 
provide  students  immediate  access  to  a wide  variety  of  full-text  journals  as  well  as  e-mail,  fax  and 
regular  mail  access  to  everything  else.  (Gibson  & Herrera,  1997,  p.  5) 

Compared  to  Berge’s  and  Welsh’s  guidelines,  Gibson  and  Herrera’s  guidelines  are  more  detailed  and 
specific,  but  also  more  situational.  For  example,  if  a WBI  course  is  focusing  on  drill  and  practice  (e.g.,  CMI 
delivered  through  the  Web),  it  might  not  need  to  have  an  online  library.  Some  of  these  guidelines  (e.g.,  number 
seven)  are  also  more  product-oriented.  These  guidelines  can  benefit  from  further  evaluation,  and  improvements  can 
be  made  to  provide  more  guidance. 

This  review  of  literature  reveals  that  current  guidance  systems  for  WBI  design  and  development  are  too 
general  and  lack  detailed  guidance.  Some  of  the  guidelines  are  also  descriptive  and  do  not  offer  detailed  instructions 
about  how  to  “do”  it.  For  a practitioner  who  wants  to  create  a WBI  course  in  the  context  of  music  education  at  pre- 
college level,  the  general  guidance  systems  reviewed  above  simply  cannot  provide  sufficient  guidance.  Therefore, 
these  general  guidance  systems  can  benefit  from  more  empirical  case  studies  for  further  refinement  and 
improvement,  especially  for  a specific  context. 

Methodology 

Phase  I:  Synthesis  of  WBI  Design  and  Development  Guidance  Systems 

There  were  two  phases  in  this  study:  an  initial  synthesis  which  focuses  on  theory  creation,  and  formative 
research  which  focus  on  theory  improvement.  During  the  phase  of  initial  synthesis,  all  of  the  guidelines  from 
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different  WBI  design  and  development  guidance  systems  were  kept  in  their  original  form  but  were  organized  into 
four  categories:  technology,  pedagogy,  implementation,  and  others.  Similar  guidelines  were  also  grouped  together 
for  easier  comparison.  The  initial  organization  was  suggested  by  one  of  the  developers  to  facilitate  the  brainstorming 
process  during  the  formal  interview  sessions.  This  organization  later  was  revised  (as  suggested  by  Reigeluth)  based 
on  chronological  order  as  we  thought  this  would  help  the  practitioners  more. 

Phase  II:  Naturalistic  Formative  Research  Study  and  Rationale 

The  purpose  of  this  phase  was  to  try  to  suggest  improvements  for  the  synthesized  guidance  system  based  on 
the  empirical  evidence  gathered  in  a single  case  study,  the  context  being  in  the  teaching  of  music  fundamentals  at 
the  pre-college  level.  The  formative  research  methodology  was  chosen.  Formative  research  is  a type  of 
developmental  research  or  action  research  that  is  intended  to  improve  a design  theory  for  instructional  practices  or 
processes  (English  & Reigeluth,  1996;  Reigeluth  & Frick,  1999).  It  was  originally  derived  from  formative 
evaluation,  which  has  the  primary  goal  of  improving  an  instructional  product  while  it  is  being  developed  in  order  to 
achieve  the  objectives  for  which  it  was  designed  (Beyer,  1995;  Dick  & Carey,  1996).  Reigeluth  and  Frick  (1999) 
further  point  out  that  “for  an  applied  field  like  education,  design  theory  is  more  useful  and  more  easily  applied  than 
its  descriptive  counterpart,  learning  theory”  (p.  633).  The  focus  of  formative  research,  therefore,  is  to  improve  a 
design  theory  (instructional  theory)  and  to  provide  detailed  prescriptive  guidance. 

Results 

After  comparing  and  triangulating  the  developers’  comments  with  other  data  sources  plus  my  own 
synthesis  and  observations,  a revised  set  of  guidelines  was  generated.  This  revised  set  of  guidelines  was  then  sent  to 
each  developer  for  further  comments  and  elaboration.  Some  of  these  guidelines  were  shifted  to  different  phases 
based  on  developers’  further  comments,  but  no  new  guidelines  were  added.  The  following  guidelines  are  the  final 
results.  Note  that  <fNew”  stands  for  new  guideline  generated  from  the  formative  research,  “Mod”  stand  for  modified 
guidelines  (from  original  source),  and  “Orig”  stand  for  original,  unchanged  guidelines. 

[Analysis  and  Planning  Phase] 

• Evaluate  all  possible  instructional  solutions.  (New) 

o Conduct  a survey  to  see  if  other  schools  have  similar  instructional  problems  and  see  if  they 
already  found  a good  solution  to  the  problem. 

o If  other  schools  have  found  a good  solution,  evaluate  their  solution  to  see  if  it  fits  into  your  own 
situation.  If  no  schools  have  a good  solution,  evaluate  other  possible  solutions, 
o Justify  the  technologies  or  solutions  you  choose, 
o Analyze  cost/benefit  issues  beforehand. 

• Assess  the  readiness  of  the  community.  (New) 

° Make  sure  that  the  intended  audience  has  the  proper  equipment  and  Internet  connections  to  access 
the  WBI  course. 

o Make  sure  that  the  intended  audience  will  accept  WBI  as  an  instructional  approach, 
o Make  sure  that  the  host  institution  has  the  proper  network  infrastructure  to  support  the  WBI 
course. 

• Secure  in  advance  the  financial  resources  to  pay  your  developers.  (Mod) 

° Develop  a detailed  budget  to  accurately  anticipate  costs. 

• Get  support  from  faculty  members  and  other  stakeholders.  (Mod) 

• Identify  enthusiastic  faculty  champions  right  away,  and  get  them  involved  with  the  project.  (Mod) 
o Choose  only  full  time  faculty  at  the  outset. 

• Decide  upfront  whether  to  implement  an  entire  course  online  or  just  selected  lessons.  (Mod) 

• Use  an  existing  course  of  studies  for  your  curriculum  development  if  possible.  (Mod) 

• Start  by  developing  small  modules  and  test  them  at  early  stages  of  development.  (Mod) 

• Conduct  a task  analysis,  and  list  all  required  tasks  in  as  much  detail  as  possible.  (New) 

• Treat  your  developers  as  a team;  hold  frequent  meetings.  They  need  to  share  ideas  and  help  each  other 
stay  focused.  There  is  much  frustration  during  the  learning  curve.  Reinforce  their  work  and  recognize 
their  accomplishments  at  every  opportunity.  (Orig) 

• Make  sure  that  the  project  director  knows  about  current  computer/web-based  technology.  If  not,  find 
an  interface  person  who  can  explain  things  to  the  director  and  act  as  a bridge.  (New) 
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• The  project  director  should  be  one  of  the  developers  or  part  of  the  development  team  if  possible.  (New) 

• Make  sure  that  all  developers  are  familiar  with  the  content  and  have  experience  teaching  with  the 
content  area  if  possible.  (New) 

[Design  Phase] 

• Write  down  your  instructional  objectives  in  detail  and  list  resources  required.  (Mod) 

• When  possible,  use  an  existing  course  as  a model  to  develop  your  instructional  modules  to  speed  up 
the  design  process.  (Mod) 

• Conceptualize  a course  as  a series  of  individual  modules,  with  each  module  comprised  of  a series  of 
instructional  events.  (Orig) 

• Sequence  performance  objectives  and  chunk  them  into  a series  of  instructional  modules,  each  of  which 
results  in  students  meeting  objectives.  When  possible,  employ  parallel  structuring  to  help  establish  a 
comfortable  rhythm  for  the  students.  (Orig) 

• For  each  event,  specify  appropriate  technology  to  enable  the  event.  Care  should  be  taken  to  choose 
technologies  available  to  all  students.  (Mod) 

• Create  a safe,  non-threatening,  and  reliable  online  learning  environment  for  the  learners, 
o Make  sure  the  learners  feel  comfortable  performing  at  an  early  stage  of  learning. 

o Use  early  encouragement  and  reassurance  to  help  the  learner  feel  comfortable  about  making  initial 
mistakes. 

o Build  up  trust  between  learners  and  the  online  learning  system.  The  system  should  be  reliable 
enough  so  that  students  can  trust  the  online  learning  technology  and  don’t  have  to  worry  about 
losing  their  completed  tasks. 

• Engage  in  formative  evaluation  and  pilot  testing  as  necessary  to  verify  that  each  event  as  well  as  the 
course  as  a whole  is  robust  pedagogically  and  procedurally.  (Orig) 

• Define/describe  and  list  the  purpose(s)  for  each  activity  level  and  the  type  of  social  and  instructional 
interactivity  and  feedback  that  is  desired.  (Orig) 

• Define  the  levels  of  teacher/computer  control,  student  control,  and  group  control  that  are  desired 
regarding  each  activity.  (Mod) 

[Development  Phase] 

• Build  up  a knowledge-sharing  and  proactive  working  culture  and  promote  innovation  in  your 
development  team.  (New) 

• Evaluate  and  choose  adequate  course  authoring/development  tools  at  early  development  stages.  (New) 

• Hire  courseware  developers  who  are  familiar  with  the  course  authoring/development  tools  you  choose. 
(New) 

• Build  a simple  group  Intranet  to  share  design  documents  among  team  members.  (New) 

• Use  an  Instant  Messenger  program  (or  other  communication  tool)  when  necessary  for  better  team 
communication.  (New) 

• Subscribe  to  or  monitor  newsgroups/listservs  of  the  development  tools  you  are  using.  (New) 

• Make  sure  that  staff  engagement  and  commitment  are  happening  in  your  team,  as  it  is  very  important 
to  the  success  of  the  WBI  design  and  development  process.  (New) 

• Use  the  minimum  technology  required  to  achieve  the  instructional  objectives.  (Mod) 

• Keep  the  online  media  (e.g.,  multimedia  files)  size  as  small  as  possible.  (Mod) 

• Develop  a sub-system  in  your  WBI  to  capture  each  student's  problem  solving  process.  This  sub-system 
should  be  able  to:  (New) 

o Keep  a history  of  students’  correct  answers. 

o Determine  if  mastery  has  been  reached.  The  mastery  criteria  can  be  made  of  three  parts: 

■ A minimum  number  of  problems  that  must  be  attempted 

■ A maximum  history  list  length  (often  the  same  as  minimum  number  of  problems  to  try) 

■ A minimum  percent  correct  of  the  problems  being  counted. 

• Test  your  prototype  on  multiple  platforms  and  browsers  at  early  stages  of  development.  (N) 

o Be  aware  that  web  pages  on  different  browsers  and  different  platforms  can  look  very  different, 
o Identify  any  cross-platform  compatibility  problems  as  early  as  possible. 
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• Provide  adequate  technical  support  for  students  in  a Web-based  instruction  environment.  (N) 
o Provide  email  or  telephone  technical  support  to  answer  students’  technical  questions. 

° Build  up  a knowledge  base  for  the  most  frequently  asked  technical  questions. 

Conclusions  and  Recommendations 

Conclusions 

It  was  not  possible  to  examine  all  aspects  of  WBI  design  and  development  guidelines  and  generate  a perfect 
guidance  system  by  synthesizing  several  general  WBI  design  and  development  guidance  systems  and  analyzing 
empirical  evidence  in  a single  case  study.  Therefore,  this  study  is  just  a beginning,  and  more  studies  are  needed  to 
further  confirm  and  elaborate  the  findings  in  this  study. 

First,  as  Powell  (2000)  mentioned: 

Choosing  Internet  technologies  to  deliver  training  should  occur  only  after  careful  consideration  of  a number  of 
factors.  These  factors  include  what  is  taught,  who  is  taught,  where  the  learning  takes  places,  how  the  teaching  is 
supported,  and  when  the  teaching  takes  place,  (p.  1) 

Schools  that  are  interested  in  WBI  should  analyze  every  possible  solution  before  making  the  decision,  as 
WBI  might  not  be  the  best  solution  in  a lot  of  situations  and  learning  domains.  Most  of  the  media  choices  are  done 
not  for  instructional  purposes,  but  for  implementation  purposes  (Powell,  2000). 

Every  WBI  needs  to  be  supported  by  instructional  theory.  Kulp  (1999)  identifies  three  instructional  models  and 
relates  each  to  the  learning  objectives  for  which  it  is  best  suited  (see  Table  2). 


Instructional  Model 

Learning  Objectives 

Learner  Activity 

Collaborative? 

1 

Instructor  centered 

Information  transfer 

Passively  receiving 

No 

2 

Learner  centered 

Skill  acquisition 

Actively  interpreting, 

practicing,  questioning, 

challenging,  discussing 

Yes 

3 

Learning  team  centered 

Mental  model  change 

Collaboratively  creating 
new  knowledge 

Yes 

Table  2:  Instructional  models  and  learning  objectives  (Kulp,  1999) 


According  to  Kulp  (1999),  the  most  common  genres  in  WBI  courses  are  learner-centered  topics.  Students  interact 
with  material  in  the  course  web  site  in  somewhat  the  same  way  they  would  with  a computer-based  training  (CBT) 
self-study  course  (Kulp,  1999).  They  actively  interpret  information  and  experience  in  order  to  create  new  knowledge 
or  build  new  work  products  of  some  type.  Based  on  this  assumption  and  Gordon’s  “Music  Learning  Theory,”  we 
can  hypothesize  that  most  of  the  guidelines  supported  by  this  case  can  also  be  useful  in  creating  other  skill-based 
WBI  courses,  although  more  research  is  needed  to  confirm  this  hypothesized  generalization.  For  other  skill-based 
pre-college  remedial  WBI  courses,  the  guidelines  provided  in  this  study  might  also  be  useful. 

Secondly,  in  this  study,  not  many  guidelines  were  rejected.  This  indicates  that  most  guidelines  were  useful 
based  on  the  empirical  evidence  for  this  case.  Some  guidelines  were  refined  to  better  fit  in  with  this  context  of 
creating  a WBI  course  at  the  pre-college  level.  One  of  the  major  findings  in  this  study  is  additional  guidelines  and 
further  elaboration  of  the  original  synthesized  guidance  system  in  this  specific  context.  Also,  some  “holes”  in  the 
original  synthesized  guidance  system  were  found,  and  new  guidelines  based  on  the  experience  from  this  case  study 
were  identified  to  fill  those  holes.  But  note  that  in  a single  case,  there  can  be  no  evidence  for  further  generalization; 
therefore  the  results  cannot  be  generalize  beyond  this  case.  Only  hypothesized  generalizations  can  be  offered,  as 
mentioned  above. 

Finally,  As  Reigeluth  and  Frick  (1999)  pointed  out: 

It  should  be  patent  that  the  development  and  testing  of  design  theories  is  not  a one-trial  endeavor.  It  is  a 
matter  of  successive  approximations.  Such  theories  continue  to  be  improved  and  refined  over  many 
iterations  (p.  635). 

Further  studies  are  needed  to  further  refine  and  elaborate  the  findings  generated  in  this  study. 
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Recommendations 

The  recommendations  in  this  study  fall  into  two  categories:  recommendations  for  future  WBI  practitioners 
and  recommendations  for  further  study.  For  developers,  I recommend  to  apply  guidelines  supported  by  this  study 
that  best  fit  in  with  your  own  project  and  to  apply  them  with  caution.  Even  though  I pointed  out  the  contextual 
influence,  and  the  elaborated  and  new  guidelines  in  this  study  look  promising,  this  is  just  one  case  study.  For  a 
single  case  study,  generalization  can’t  really  be  made  beyond  the  context  of  music  fundamentals  instruction  because 
of  lack  of  other  research  to  support  those  generalizations.  Further  studies  are  needed  to  confirm  those  findings 
before  we  really  have  confidence  in  their  generalizability. 

Further  study  is  recommended  to  continuously  elaborate  and  refine  the  WBI  design  and  development 
guidelines  for  the  context  of  teaching  music  fundamentals  at  the  pre-college  or  college  level.  As  this  study  only 
covered  the  design  and  development  phases,  there  is  still  plenty  of  room  for  further  elaboration  and  refinement  in 
this  context,  especially  to  include  guidelines  for  the  implementation  and  evaluation  phases.  Secondly,  I recommend 
future  research  to  address  other  learner  age  groups  and  the  content  areas.  For  example,  WBI  might  be  a good 
solution/medium  for  pre-college  or  college  learners,  but  how  about  younger  or  older  learners?  Also,  how  about  other 
types  of  music  courses  such  as  music  history  or  advanced  music  theory  courses?  Will  the  WBI  design  and 
development  guidelines  concerning  these  different  content  domains  or  age  groups  be  different  from  the  guidelines 
provided  in  this  study?  Further  research  in  this  area  can  help  us  get  a sense  of  the  generalizability  of  the  guidelines 
supported  by  this  study.  Finally,  the  product  of  the  MFO  project--the  WBI  course-- also  needs  some  research.  At  the 
time  of  this  writing,  the  MFO  team  was  just  about  to  start  their  larger-scale  final  beta  testing  of  its  prototype  and  will 
have  more  revisions  in  the  future.  So  further  formative  research  on  the  ongoing  MFO  project  is  highly  recommended 
to  generate  product-oriented  guidelines.  This  is  an  important  part  of  my  research  agenda  for  the  near  future.  Other 
recommended  product-oriented  research  directions  include: 

1.  Learning  styles  and  WBI:  Learning  styles  deal  with  characteristic  styles  of  learning.  Kolb  (1984)  proposes  a 
theory  of  experiential  learning  that  involves  four  principal  stages:  concrete  experiences,  reflective  observation, 
abstract  conceptualizations,  and  active  experimentation.  He  also  postulates  four  types  of  learners:  assimilators, 
di vergers,  convergers,  and  accommodators.  For  example,  Kolb  (1984)  points  out  that  an  accommodator  prefers 
concrete  experiences  and  active  experimentation.  Pask  (1988)  also  has  described  two  learning  styles:  serialist 
and  holist.  According  to  Pask  (1988),  serialists  prefer  to  learn  in  a sequential  fashion,  whereas  holists  prefer  to 
learn  in' a hierarchical  manner.  As  WBI  is  more  flexible  and  powerful  in  the  way  that  it  can  provide  more 
individualized  or  personalized  instruction,  it  will  be  interesting  to  conduct  research  to  investigate  what  kind  of 
instructional  strategies  or  approaches  would  be  most  effective  in  a WBI  product  and  their  relationship  with 
different  cognitive  and  learning  styles. 

2.  Music  aptitude  and  WBI:  In  the  past,  musical  ability  was  often  viewed  in  all-or-none  terms:  some  are  blessed 
with  "talent,"  others  must  do  without.  Recent  research  (Gordon,  1993;  Baney,  1999),  however,  reveals  that 
music  aptitude,  like  all  human  characteristics,  is  normally  distributed  in  the  population.  Relatively  few  have 
high  aptitude,  a similar  number  have  low  aptitude,  and  the  majority  of  persons  fall  somewhere  in  the  middle  of 
the  "bell  curve"  with  average  aptitude.  In  another  words,  most  persons  have  the  potential  to  achieve  in  music.  It 
will  be  interesting  to  see  how  people  with  different  music  aptitudes  interact  with  Web-based  courses  such  as 
MFO.  More  research  is  recommend  to  find  out  the  relationship  between  music  aptitude,  music  learning  theory 
(Gordon,  1997),  and  WBI. 
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Concept  Learning  Through  Image  Processing 


Lauren  Cifuentes 
Yi-Chuan  Jane  Hsieh 
Texas  A&M  University 


Abstract 

This  study  explored  computer-based  image  processing  as  a study  strategy  for  middle-schoolers  ’ science 
concept  learning.  Specifically,  researchers  examined  the  effects  of  computer  graphics  generation  on  science  concept 
learning  and  the  impact  of  using  computer  graphics  to  show  interrelationships  among  concepts  during  study  time. 
Educators  are  encouraged  to  prepare  learners  to  use  computers  to  visualize  concepts  during  study  time.  An 
orientation  to  visualization  skills  can  prepare  students  for  using  visual  techniques  to  represent  interrelationships 
among  concepts. 

Background  and  Theoretical  Perspective 

Computer  graphics  software  such  as  AppleWorks™  and  PhotoShop™  have  become  pervasive  in  today’s 
schools.  Such  software  allow  students  to  access  a variety  of  tools  that  help  them  draw  and  paint  objects  to  visually 
organize  and  represent  what  they  know.  Student-generated  interpretative  illustrations  can  help  clarify  the  profound 
concepts  expressed  in  texts  and  facilitate  the  comprehension  of  abstract  concepts.  When  students  are  able  to 
manipulate  images  during  knowledge  construction,  they  tend  to  engage  more  in  the  meaning-making  process  and 
understand  and  remember  concepts  better  than  through  the  traditional  transmission  approach  of  instruction 
(Jonassen,  2000).  Additionally,  students’  graphic  representations  of  what  they  know,  can  provide  products  for 
teacher  feedback. 

One  approach  for  knowledge  construction  commonly  used  by  students  is  concept  mapping  (Anderson- 
Inman  & Ditson,  1999).  Concept  mapping  refers  to  the  “process  for  representing  concepts  and  their  relationships  in 
graphical  form,  providing  students  with  a visually  rich  way  to  organize  and  communicate  what  they  know”  (p.7). 
Research  has  suggested  that  students  can  study  efficiently  by  generating  concept  maps  (Anderson-Inman  & Zeitz, 
1993).  Also,  Cifuentes  (1992)  found  that  students  who  visualized  interrelationships  among  concepts  in  their  hand 
written  study  notes  performed  significantly  higher  on  a test  ( p=  .02)  than  those  students  who  did  not  show  such 
interrelationships.  Concept  mapping  can  be  fostered  through  computer-based  software  such  as  Inspiration™. 
Additionally,  images  can  be  manipulated  to  show  interrelationships  with  image-processing  software  such  as 
AppleWork’s™  drawing  and  painting  tools.  These  visualization  tools  can  be  regarded  as  “mindtools”  to  extend  and 
reorganize  learners’  cognitive  structures  during  learning  (Jonassen,  2000).  Learners  can  use  graphic  conventions  for 
organizing  their  thoughts  as  they  construct  knowledge  of  concepts  (Dodge,  1998).  Computer-generated  graphics 
created  by  learners  offer  several  advantages  over  pen  and  paper  such  as  ease  of  subsequent  revision  and  generation 
of  sophisticated  looking  graphics  by  students  with  undeveloped  artistic  skill. 

Objectives 

This  study  explored  computer-based  image  processing  as  a study  strategy  for  middle  school  science 
concept  learning.  Specifically,  researchers  examined  the  effects  of  computer  graphics  generation  on  science  concept 
learning  and  the  impact  of  using  computer  graphics  to  show  interrelationships  among  concepts  during  study  time. 

Methods 

The  87  students  engaged  in  this  study  were  volunteers  taking  junior  high  school  science  classes  at  a rural, 
public,  junior  high  school.  They  had  previous  skill  with  AppleWorks™  draw  and  paint  software. 

Quantitative  and  qualitative  methods  were  used  to  investigate  the  research  topic.  Potential  participants  were 
the  entire  7th  and  8th  grade  student  body  of  a rural  school  in  Texas.  However,  some  of  those  students  did  not  turn  in 
consent  forms,  some  were  absent  for  part  of  the  treatment,  and  others  were  absent  for  testing.  Therefore,  87  students 
participated  in  the  complete  study.  Science  classes  were  randomly  divided  in  half  from  both  the  seventh  and  eighth 
grades  so  that  approximately  half  of  the  student  body  were  assigned  to  the  control  group  (n=  46)  and  approximately 
half  of  the  student  body  were  assigned  to  the  experimental  group  (n  = 41).  The  groups  were  comparable  across  age, 
gender,  and  ethnicity.  The  control  group  consisted  of  50%  seventh  graders  and  50%  eighth  graders.  59%  were  male 
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and  41%  were  female.  54%  percent  were  Caucasian,  24%  percent  were  African-American,  and  22%  percent  were 
Hispanic.  The  experimental  group  consisted  of  44%  seventh  graders  and  56%  eighth  graders.  49%  were  male  and 
51%  were  female.  51%  percent  were  Caucasian,  34%  percent  were  African-American,  10%  percent  were  Hispanic, 
and  5%  identified  themselves  as  a mixture  of  races.  Participants  were  assigned  to  one  of  two  groups,  one  receiving 
treatment  and  one  for  control,  in  a post-test-only-control-group  design.  Posttest  scores  were  compared  across  groups. 
In  addition,  student’s  study  strategies  were  compared  qualitatively  to  help  explain  the  impact  of  graphics  on 
learning.  The  four  data  sources  included:  (a)  immediate  recall  test,  (b)  students’  study  notes,  (c)  students’  computer 
files,  and  (d)  a Web-based  “Study  Strategies  Questionnaire”. 

Quantitative  aspect:  All  participating  science  courses  were  placed  in  a hat  and  drawn  to  randomly  assign 
them  to  the  two  treatment  groups: 

♦ Control  group--  received  print-based,  verbal  material  on  a science  concept  (General  Properties  of 
Matter)  and  were  given  50  minutes  for  unguided,  independent  study  prior  to  the  test.  Students  had 
access  to  computers  during  study  time.  At  the  end  of  the  50  minutes  students  kept  both  their  study 
notes  and  reading  material.  The  next  day,  and  prior  to  taking  the  test,  students  handed  in  their  study 
notes. 

♦ Experimental  group — participated  in  a workshop  consisting  of  three  50-minute  training  sessions  on 
how  to  manipulate  and  generate  computer  graphics  during  study  time  using  materials  developed  by  the 
researchers  in  AppleWorks™,  Photoshop™,  and  PowerPoint™.  They  then  received  the  same  print- 
based,  verbal  material  that  the  control  group  received.  It  was  on  a science  concept  (General  Properties 
of  Matter)  and  students  were  given  50  minutes  to  study  prior  to  a test.  At  the  end  of  the  50  minutes 
students  kept  both  their  study  notes  and  reading  material.  The  next  day,  and  prior  to  taking  the  test, 
students  handed  in  their  study  notes. 

The  three  50-minute  workshop  sessions  on  how  to  manipulate  and  generate  computer  graphics  during  study 
time  had  the  following  objectives:  for  students  to  be  able  to  (a)  recognize  underlying  structure  of  text 
(interrelationships),  (b)  illustrate  underlying  structure,  (c)  relate  new  concepts  to  prior  knowledge,  (d)  highlight 
distinctive  features,  and  (e)  use  graphics  for  review.  The  researchers  facilitated  the  workshop.  They  modeled 
visualization  of  concepts  using  7 underlying  structures,  gave  the  students  a turn  with  each  of  the  7 underlying 
structures,  modeled  direct  representation  of  concepts  and  highlighting  distinctive  features,  gave  the  students  a turn 
with  direct  representation  and  highlighting,  and  gave  students  15  short  texts  to  visualize.  They  encouraged  students 
to  keep  their  graphic  representations  and  use  them  for  study  and  review. 

After  taking  the  immediate  recall  test,  participants  filled  in  a Web-based  “Study  Strategies  Questionnaire” 
that  asked  them  to  rate  the  extent  that  they  had  previously  been  exposed  to  the  information  in  “General  Properties  of 
Matter?”  To  determine  if  groups  varied  in  their  prior  knowledge  of  the  textual  material,  a t-test  was  conducted.  No 
difference  was  found.  The  questionnaire  also  asked  students  to  describe  in  detail  the  steps  that  they  took  to  prepare 
for  the  test.  The  testing  instrument  for  immediate  recall  contained  30  multiple-choice  items.  All  students  took  the 
immediate  recall  test  at  the  end  of  their  50-minute  study  to  determine  the  effects  of  the  experimental  treatment. 

In  addition,  all  participants’  study  notes  and  printouts  of  computer  graphics  were  collected.  Participants 
were  asked  on  the  Web-based  survey  to  describe  in  detail  the  steps  that  they  took  to  prepare  for  the  test.  The 
researchers  rated  the  participants  as  visualizers  or  nonvisualizers  based  upon  the  students’  study  notes  and  study 
strategies  reported  on  the  survey.  We  classified  students  as  visualizers  if  they  used  the  computers  to  construct 
visuals  while  they  studied  for  the  test  and/or  reported  that  they  hightlighted  or  drew  while  they  studied.  We 
classified  students  as  nonvisualizers  if  they  did  not  create  visuals  or  highlight  during  study.  The  effects  of  treatment 
and  application  of  visualization  during  study  time  on  immediate  recall  were  then  estimated  by  comparing  scores 
using  planned  contrasts  in  a general  linear  model. 

The  design  avoided  effects  of  initial  bias,  previous  testing,  maturation,  instrumentation,  regression, 
selection,  and  mortality  because  groups  were  randomly  assigned  and  were  not  pretested. 

Qualitative  aspect: 

We  applied  content  analyses  approaches,  as  described  by  Emerson,  Fretz,  and  Shaw  (1995),  to  the  study 
notes,  computer  files,  and  survey  results.  During  and  upon  completion  of  data  collection,  we  used  the  two-phase 
process  of  content  analyses,  open  coding  and  focused  coding. 
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Results 

The  ANOVA  revealed  a negative  treatment  effect.  The  control  group  performed  better  than  the 
experimental  group  and  there  was  no  statistical  difference  between  the  scores  of  visualizers  and  nonvisualizers.  In 
addition,  there  was  no  interaction  between  group  and  strategy  on  students’  performance. 

In  this  case,  computer-based  image  processing  was  not  an  effective  study  strategy  for  science  concept 
learning.  Middle  school  students  were  unable  to  identify  or  represent  underlying  structure.  They  claimed  that 
visualization  was  too  hard  for  them  and  they  expressed  lack  of  motivation.  They  were  distracted  by  the  computers 
and  the  fun  software  and  were  distracted  by  graphics  tools.  They  spent  their  time  creating  irrelevant  images  and 
generated  visuals  that  involved  inefficient  use  of  time. 

The  students  who  received  the  workshop  in  visualization  as  a study  strategy  did  not  perform  better  on  the 
test  on  “The  General  Properties  of  Matter.  Several  environmental  factors  affected  the  impact  of  the  workshop. 
Workshop  time  was  insufficient  and  students  had  difficulty  internalizing  the  visualization  as  a study  strategy  while 
they  studied.  Students  were  unable  to  manage  their  time  for  studying  the  entire  text  and  meanwhile  draw  meaningful 
visuals  to  foster  their  understandings. 

In  addition,  the  students  who  visualized  concepts  while  studying  did  not  perform  better  on  the  test.  They 
often  spent  time  visualizing  what  they  already  knew  rather  than  grappling  with  a new  concept.  The  visuals  generated 
by  students  were  often  inappropriate  or  misrepresented  concepts  and  therefore  could  not  facilitate  concept  learning 
(see  Tables  1 and  2). 


Table  1 

Descriptive  Statistics  for  Groups 


Group 

Strategy 

Mean 

Std.  Deviation 

N 

Control 

Nonvisualizers 

14.11 

5.48 

36 

Visualizers 

14.00 

4.57 

10 

Total 

14.09 

5.25 

46 

Experimental 

Nonvisualizers 

10.33 

4.56 

18 

Visualizers 

13.30 

3.30 

23 

Total 

12.00 

4.13 

41 

Total 

Nonvisualizers 

12.85 

5.45 

54 

Visualizers 

13.52 

3.67 

33 

Total 

13.10 

4.84 

87 

87 
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Table  2 

ANOVA  for  Effects  of  Treatment  and  Visualization 


Source 

Type  III  Sum  of 
Squares 

df 

Mean 

Square 

F 

Sig. 

Group 

88.230 

1 

88.230 

3.996 

.049* 

Strategy 

36.061 

1 

36.061 

1.633 

.205 

Group  * Strategy 

41.883 

1 

41.883 

1.897 

.172 

Error 

1832.425 

83 

22.077 

Total 

16954.000 

87 

R Squared  = .091  (Adjusted  R Squared  = .058)  * significance  /?<_*05 


We  delivered  the  workshop  in  a naturalistic,  rural  school  setting.  Therefore,  many  unexpected 
environmental  occurrences  affected  the  delivery  of  the  workshop.  For  instance,  we  encountered  electrical  failure  in 
the  projector,  and  were  unable  to  model  visualization  processes  on  the  presenter  computer.  We  had  to  adopt  “plan 
B”  and  use  overhead  transparencies.  Of  course,  this  meant  that  we  were  unable  to  model  using  graphics  tools  for 
visualization.  All  during  the  workshop,  electricians  and  teachers  came  in  and  out  of  the  computer  lab  to  address  the 
technological  problems,  or  students  would  ask  to  go  to  get  water  or  go  to  the  restroom.  Some  students  talked  and 
visited  with  each  other  rather  than  focus  on  learning  and  some  did  not  participate  in  parts  of  the  workshop  because 
of  illness,  extracurricular  activities,  or  loss  of  interest.  Additionally,  the  arrangements  of  the  projector  and  students’ 
workstations  (chairs,  tables,  computers)  were  so  inharmonic  that  students’  positions  had  to  shift  to  look  at  the 
presenter,  projected  images,  and  their  computer  screens  alternately.  Furthermore,  because  several  computers  in  the 
lab  were  out  of  order,  many  students  had  to  share  computers  with  each  other.  Similarly,  access  to  printers  was 
limited.  Only  four  out  of  the  twenty  computers  were  connected  to  the  classroom  printer,  which  made  the 
preservation  of  students’  work  difficult.  Students  could  not  incorporate  Web  graphics  into  their  visualizations 
because  none  of  the  computers  were  connected  to  the  World  Wide  Web. 

Moreover,  the  middle  schoolers  were  extremely  distracted  by  the  software  on  the  computers,  especially  the 
multimedia  authoring  software.  Students  tended  to  easily  create  sounds  and  irrelevant  images  on  their  computers.  In 
addition,  the  7th  and  8th  grade  students  spent  a lot  of  time  generating  playful  graphics  or  visuals  that  did  not  help 
them  build  their  understandings.  They  did  not  make  good  use  of  their  time  for  studying  the  entire  assigned  text. 
They  tended  to  take  the  fun  and  easy  route  by  visualizing  what  they  already  knew  rather  than  grappling  with  a new 
concept.  Several  students  said  that  they  found  identifying  the  underlying  structure  of  the  text  to  be  quite  difficult  and 
we  found  that  most  students  were  often  unable  to  create  appropriate  representations  of  new  concepts.  Most  students 
were  either  unwilling  or  incapable  of  thoroughly  and  accurately  representing  texts.  When,  for  instance,  they  were 
asked  to  represent  the  periods  included  in  the  Mesozoic  Era,  one  student  neglected  to  draw  arrows  in  her  timeline  to 
indicate  continuity  and  did  not  include  the  Mesozoic  Era  in  her  graphic.  She  also  placed  the  periods  in  the  wrong 
order  on  the  timeline  indicating  her  lack  of  understanding  (see  Figure  1).  Another  student  misrepresented  the  visual 
of  an  iceberg.  He  was  enthusiastic  about  drawing  an  iceberg,  but  ignored  the  text’s  main  idea  that  only  a small  part 
of  an  iceberg  is  above  the  waterline  and  the  rest  of  an  iceberg  is  under  water  (see  Figure  2). 

Text  to  visualize:  The  Scientists  divide  the  Mesozoic  Era  into  three  periods. 

The  oldest  period  is  called  the  Triassic  Period.  The  middle  period  is  called  the  Jurassic  Period.  The  youngest  period 
is  called  the  Cretaceous  Period. 

Cretaceous  Jurassic  Triassic 

Period  | Period  | Period 

Figure  1.  The  misrepresentation  of  chronological  information. 
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Figure  2.  The  misrepresented  concept  of  icebergs . 


On  the  other  hand,  some  workshop  participants  successfully  identified  and  visualized  the  underlying 
structure  of  the  text  they  were  studying.  For  instance,  most  students  were  able  to  successfully  represent  the  sequence 
of  the  moon  phases  as  well  as  chronology  of  periods  within  eras.  Also,  many  students  were  able  to  directly  represent 
concepts.  For  instance,  one  student  generated  a direct  representation  of  how  the  earthworm  breathes  (see  Figure  3). 


EARTHWORM 


Capillaries 


Carbon 

Dioxide 


<£=■  (7T7T7 

Oxygen 


Figure  3.  Student-generated  visualization  of 
the  earthworm ’s  respiratory  process. 


In  general,  the  processes  of  identification  of  the  underlying  structure  of  concepts  presented  in  texts  and 
subsequent  creation  of  visual  representations  associated  with  those  concepts  required  a lot  of  time  and  effort  from 
students.  Such  strategies  facilitate  memory  because  they  involve  the  learner  at  a high  level  of  cognitive  processing 
by  demanding  extensive  learner-effort.  The  visualization  process  helped  most  of  our  participants  think  hard  about 
what  they  needed  to  learn,  and  learning  that  was  not  measured  on  the  test  resulted  from  such  thinking.  When 
students  were  able  to  extract  meaning  from  text  and  generate  representative  images,  they  built  their  complete 
understandings.  For  example,  as  one  student  tried  to  understand  that  weight  of  an  object  changes  according  to 
altitude,  but  that  mass  remains  the  same,  he  generated  a computer  graphic  accurately  conveying  an  object  with  the 
same  mass  on  a mountaintop  and  in  a deep  mine.  Such  kinds  of  visualizations  concretized  what  students  cognitively 
comprehended,  and  helped  them  clarify  meaning  of  science  concepts  (see  Figure  4). 
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You  have  the  same  mass  on  top  of  a 
mountain  as  you  do  in  a deep  mine. 


Figure  4.  Student-generated  computer  graphic 


Educational  Significance 

We  discovered  that  learner  generated  graphic  representations  of  concepts  provided  a rich  resource  for  the 
students’  teachers.  Representations  of  learner’s  understandings  provided  teachers  with  a way  of  knowing  whether  or 
not  students  grasp  concepts.  Teachers  suggested  that  if  students  can  not  visualize  the  concept,  perhaps  they  don’t 
thoroughly  understand  the  concept. 

In  summary,  we  delivered  a visualization  workshop  designed  to  help  learners  use  computer  graphics  to 
construct  meaning  while  they  study.  When  delivering  visualization  workshops  in  the  natural  setting  of  schools, 
problems  are  bound  to  arise.  Technical  failure,  human  interruptions,  lack  of  active  participation,  limited  access, 
distraction  by  alternate  tools  or  games  on  computers,  and  differences  in  learners’  abilities  each  affect  the  success  of 
a workshop.  Middle  schoolers  are  unsophisticated  learners  and  require  guidance  toward  effective  visualization.  In 
our  workshop,  in  spite  of  problems  associated  with  the  natural  setting  of  a school,  students  engaged  actively  in  the 
meaning-making  process  of  studying  while  we  provided  scaffolding.  Students  who  successfully  formulated  mental 
representations  of  concepts  and  then  concretized  those  representations  as  computer  graphics  applied  a strategy  for 
spending  time  thinking  in  order  to  learn. 

Still,  they  require  cognitive  apprenticeship  and  expert  modeling  of  identifying  the  underlying  structure  of 
concepts.  Given  the  3-day  workshop,  students  were  unable  to  internalize  the  visualization  methods  as  part  of  their 
study  strategy.  Most  students  were  cognitively  not  ready  to  generate  meaningful  computer  graphics  while  they  study 
the  textual  information.  They  were  more  likely  to  highlight  the  important  points  by  typing  those  words  or  sentences 
on  computer  than  producing  visuals  concretizing  those  concepts.  Middle  schoolers  might  need  extensive  practice  in 
constructing  their  own  concept  representations  while  receiving  expert  feedback  regarding  their  appropriateness 

The  7th  and  8th  grade  students  who  participated  in  our  workshop  were  not  sophisticated  visualizers.  They 
needed  expert  modeling  of  identifying  underlying  structure  of  texts.  It  is  essential  to  have  all  students  engaged  in  the 
diverse  practice  of  constructing  their  own  concept  representations  while  receiving  expert  feedback  regarding  their 
appropriateness.  Additionally,  the  instructor  should  teach  students  in  the  sequence  of  increasingly  complex  tasks. 
When  the  learning  task  becomes  more  and  more  difficult  for  students  to  handle  independently,  the  aids  of  peers  and 
the  close  scaffolding  of  the  instructor  are  of  great  importance. 
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A Connectionist  Model  of  Instructional  Feedback  Effects 


Roy  B.  Clariana 
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Abstract 

This  experimental  investigation  provides  empirical  support  for  the  potential  of  a connectionist  approach  to 
predict  instructional  feedback  effects.  Posttest  data  from  high  school  students  is  compared  to  values  predicted  by 
the  delta  rule.  Results  and  implications  are  provided. 

Learning  involves  the  interaction  of  information  provided  by  instruction  with  existing  information  already 
in  the  learner's  memory  (Ausubel,  1968;  Bruner,  1990).  When  a learner  commits  to  a lesson  response,  that  response 
reflects  the  learner’s  immediate  understanding  of  that  instructional  instance,  thus  initial  lesson  response  (ILR) 
provides  a measure  of  a learner's  existing  information. 

Describing  what  happens  to  memory  traces  of  ILRs  that  are  errors  is  necessary  for  determining  whether 
errors  interfere  with  attaining  correct  responses,  and  so  is  one  key  to  understanding  how  feedback  works.  Clariana 
(1999)  has  suggested  that  a connectionist  approach  using  the  delta  rule  can  be  used  to  predict  posttest  memory 
activation  levels  of  ILR  errors  and  of  correct  responses  for  immediate  and  delayed  feedback.  This  investigation 
provides  data  that  supports  his  approach.  But  first,  what  is  a connectionist  approach? 

Connectionist  Description  of  Feedback  Timing  Effects 

Connectionist  models  apply  various  mathematical  rules  within  neural  network  computer  simulations  in  an 
effort,  among  other  things,  to  mimic  and  describe  human  memory  associations  and  learning.  The  theory  includes 
several  families  of  models,  such  as  simple  feedforward  networks,  pattern  associators,  multi-layer  networks  with 
backpropagation,  competitive  networks,  and  recurrent  networks,  that  differ  slightly  in  how  the  nodes  of  the  network 
are  interconnected,  but  which  differ  greatly  in  the  kind  of  processing  that  they  are  able  to  do  (see  McLeod,  Plunkett, 
& Rolls,  1998,  for  more  detail).  Neural  networks  have  been  shown  to  be  capable  of  pattern  matching,  pattern 
completion,  retrieval  by  content,  recognition,  prototype  extraction,  and  classification  to  name  a few  (Haberlandt, 
1997). 

For  example,  Seidenberg  and  McClelland  (1989)  trained  a computer  neural  network  to  read  aloud  all 
English  monosyllabic  words  (about  3,000  words).  After  250  training  epochs,  the  model  could  correctly  pronounce 
97%  of  the  3,  000  words  in  the  training  set.  This  neural  network  was  able  to  accomplish  this  task  without  a local 
lexical  store  and  more  importantly  without  being  given  a set  of  rules.  Elman  (1993)  trained  a neural  network  with 
sentences  rather  than  words,  and  was  able  to  show  that  the  network  could  satisfy  long-distance  grammatical 
dependencies  (matching  syntax).  Plunkett  and  Marchman  (1993;  1996)  have  modeled  early  lexical  development 
which  parallels  that  observed  in  children.  Their  neural  network  model  that  produces  past  tense  forms  of  regular  and 
irregular  verbs  has  challenged  the  language  acquisition  orthodoxy  that  language  learning  depends  on  both  innate 
pre-wiring  of  the  system  and  on  learning  symbolic  rules  of  the  language. 

Among  a number  of  connectionist  learning  rules,  the  delta  rule  (Shanks,  1995;  Widrow  & Hoff,  1960)  is 
one  of  the  simplest  and  most  common  that  includes  the  effects  of  feedback  on  learning.  The  delta  rule  describes  the 
change  in  association  weight,  termed  Aw,  between  an  input  unit  and  an  output  unit  at  each  learning  trial,  as: 

Awi0  = a ain  (t0  - a^t) 

where  a is  the  learning  rate  parameter,  ajn  is  the  activation  level  of  input  units,  t*,  is  the  desired  response  (the  t refers 
to  "teacher",  in  this  case  ^ is  item  feedback),  and  a^  is  the  activation  level  of  the  output  units  (Shanks,  1995).  In 
instructional  terms,  learning  is  an  increase  in  association,  that  is,  an  increase  in  Awio  between  the  stimulus  (ain)  and 
the  correct  response  (a^),*  with  a relative  decrease  in  association,  that  is,  a decrease  in  Awio  for  incorrect  responses. 

To  apply  the  delta  rule  in  this  study,  following  Clariana  (1999),  this  investigation  assumes  that  lesson 
average  item  difficulty  values  are  reasonable  estimates  of  the  association  weights  of  the  correct  responses.  Item 
difficulty  (p)  is  defined  as  the  proportion  (pg)  of  individuals  who  answer  an  item  (g)  correctly  (item  difficulty 
notation  convention  from  Crocker  & Algina,  1986).  For  example  an  item  difficulty  of  .20  indicates  that  20%  of  the 
learners  responded  correctly  to  that  item.  Item  difficulty  values  range  from  0.00  to  1.00  with  low  values  indicating 
difficult  items  and  high  values  indicating  easy  items.  Using  lesson  average  item  difficulty  values  as  a group's 
estimate  of  the  initial  lesson  item  aout  association  weight  seems  reasonable  in  that  lesson  item  difficulty  is  the  actual 
averaged  probability  of  selecting  that  alternative  as  the  correct  response  during  the  lesson  for  that  population  of 
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learners.  To  our  knowledge,  this  is  the  first  investigation  to  utilize  lesson  item  difficulty  values  as  a measure  of 
input  and  output  activation  levels. 

In  the  delta  rule  equation,  feedback  impacts  learning  in  the  term  (to  - a^).  Customarily,  the  values  for  L 
and  aoUt  are  constrained  between  0 and  1 (McLeod,  Plunkett,  & Rolls,  1998).  The  value  for  t0  equals  1 if  the 
activation  level  of  the  output  unit  matches  the  desired  response  (i.e.,  with  correct  responses)  and  t0  equals  zero  if  the 
activation  level  of  the  output  unit  does  not  match  the  desired  response  (i.e.,  with  incorrect  responses).  So  with 
correct  responses  the  association  weight  increases  since  (1  - aout)  is  positive,  while  with  incorrect  responses  the 
association  weight  decreases  since  (0  - a^t)  is  negative. 

In  other  words,  when  feedback  is  provided  as  part  of  the  responding  instance,  correct  responses  are 
strengthened  and  incorrect  responses  weakened.  The  amount  of  increment  or  decrement  can  be  determined  by  the 
delta  rule.  Thus,  given  lesson  item  difficulties  (initial  a^t),  the  delta  rule  should  be  able  to  predict  posttest  item 
difficulties  (a^t  after  feedback).  What  values  would  the  delta  rule  provide  for  ILRs  for  immediate  and  for  delayed 
feedback?  In  the  present  investigation,  first  the  delta  rule  would  predict  that  for  correct  lesson  responses,  memory  of 
ILRs  and  of  correct  responses  would  be  strengthened  in  general  for  both  immediate  and  delayed  feedback,  since  L = 
1 and  so  (t0  - aout)  is  positive.  Second,  for  lesson  errors,  the  ILR  association  with  the  item  stem  would  be  weakened 
for  immediate  feedback  since  L = 0 and  (L  - aout)  is  negative,  but  not  for  delayed  feedback. 

For  delayed  feedback,  the  connectionist  model  would  predict  that  ILR  errors  would  actually  be 
strengthened.  In  associative  learning  in  living  systems,  there  is  a small  window  of  time  while  the  specific  input 
pattern  is  activated  lasting  probably  less  than  4 seconds  (Shanks,  Pearson,  & Dickerson,  1989)  when  those 
associations  can  be  strengthened  or  weakened.  Immediate  feedback  provides  the  necessary  teacher  feedback 
information  within  this  time  frame  while  delayed  feedback  does  not.  Specifically,  with  delayed  feedback,  since 
corrective  feedback  is  not  immediately  provided,  the  learning  rule  association  process  would  act  as  though  the  error 
response  is  correct,  thus  strengthening  the  association  weight  of  the  error  (see  Figure  1). 


0.20  0.40  0.60  0.80 


Lesson  Item  Difficulty  (p) 

Figure  I . Predicted  retention  test  values  generated  by  the  delta  rule  across  a range  of  lesson  item  difficulty  values 
(from  Clariana,  1999).  Predicted  retention  test  memory  of  Initial  Lesson  Responses  (ILR)  are  shown  as  dashed 
lines  and  Correct  Responses  (CR)  are  shown  as  solid  lines  for  delayed  feedback  (DF)  and  immediate  feedback  (IF). 
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Based  on  the  connectionist  model  of  feedback  timing  described  by  Clariana  (1999),  several  hypotheses  can 
be  stated:  (1)  Retention  test  memory  of  ILRs  will  be  considerably  greater  for  delayed  feedback  than  for  immediate 
feedback  at  all  item  difficulty  levels  (see  dashed  lines  in  Figure  1).  (2)  Both  types  of  feedback  will  obtain  the 
greatest  lesson  to  posttest  gain  with  difficult  lesson  items.  (3)  Retention  test  memory  of  correct  responses  will  vary 
across  the  range  of  possible  lesson  item  difficulty  values  for  the  delayed  and  immediate  feedback  forms,  with 
immediate  feedback  slightly  better  than  delayed  feedback  with  more  difficult  lesson  items  and  delayed  feedback 
slightly  better  than  immediate  feedback  with  easier  lesson  items  (see  the  solid  lines  in  Figure  1). 

In  addition,  it  may  be  possible  to  separate  any  observed  effects  of  immediacy  versus  multiple-exposures  by 
including  a type  of  feedback,  multiple-try  immediate  feedback  (MTF),  which  has  aspects  of  both  (Bangert-Drowns, 
Kulik,  Kulik,  & Morgan,  1991;  Clariana,  1993).  MTF  is  like  the  more  common  form  of  immediate  feedback 
(single-try  feedback,  STF)  in  terms  of  immediacy  of  feedback  timing.  But  in  terms  of  multiple  explicit  stimulus 
exposures,  multiple-try  feedback  is  like  delayed  feedback  (DF),  at  least  with  lesson  errors. 

Method 

The  available  sample  for  this  study  included  students  from  three  high  school  social  studies  classes  (n=  87) 
from  a small  town  in  a northeastern  state.  The  students  were  mostly  sophomores,  with  a few  juniors.  A number  of 
students  chose  not  to  participate,  some  were  absent,  a few  forgot  to  return  their  signed  consent  forms,  and  three 
students’  data  were  dropped  due  to  incomplete  data,  yielding  a final  sample  of  52  students.  The  final  sample 
contained  more  females  (71%)  than  males. 

The  computer-based  lesson  material  consisted  of  eight  reading  passages  and  36  five-alternative  multiple- 
choice  questions  from  the  Nelson-Denny  Reading  Test,  Form  E (Brown,  Bennett,  & Hanna,  1981;  with  permission  of 
the  publisher).  These  text  passages  and  questions  were  chosen  based  on  the  quality  of  the  text  and  questions,  their 
high  reliability,  the  availability  of  extensive  test  reliability  metrics,  and  content  appropriateness  for  this  audience. 
Field  trials  by  the  developers  involved  more  than  14,000  students  (Brown,  Bennett,  & Hanna,  1981).  Currently, 
these  materials  are  extensively  used  in  the  field  and  are  readily  available  from  the  publishers. 

The  36  questions  consisted  of  18  verbatim  questions  and  18  inferential  questions,  similar  to  Peeck  and 
Tillema  (1978).  Verbatim  questions  relate  directly  to  one  proposition  in  the  instructional  text.  For  example,  the  text 
may  say,  "Daytimes  Robert  Browning  walked  with  Elizabeth’s  little  dog  Flush,  but  he  seldom  could  be  lured  from 
his  home  evenings”,  with  the  associated  verbatim  question,  "Flush  was  the  name  of  the  Brownings’:  A.  cat  B.  dog 
C.  canary  D.  gold  fish  E.  thrush?’’.  Inferential  questions  relate  to  multiple  propositions  in  the  text  and  can  be 
answered  by  considering  the  passage  as  a whole.  For  example,  the  inferential  question,  "The  authors  of  this  passage 
placed  most  attention  on  the  Brownings:  A.  literary  efforts  B.  personal  relationship  C.  social  contacts  D. 
problems  E.  early  meeting?",  can  be  answered  by  considering  the  entire  passage.  Thus,  inferential  questions  have 
many  indirect  connections  to  the  instructional  text,  while  verbatim  questions  have  one  or  only  a few  direct 
connections  to  the  text. 

The  computer-based  lessons  were  developed  in  HyperCard  and  delivered  on  Macintosh  computers.  The 
three  alternative  computer-based  lessons  were  identical  except  for  the  type  of  feedback  that  was  presented  after  the 
learner  responded  to  multiple-choice  questions.  For  all  treatments,  each  of  the  eight  text  passages  was  presented  in 
turn  in  scrolling  text  windows.  After  reading  a text  passage,  the  learner  would  respond  to  either  four  or  eight  five- 
alternative  multiple-choice  questions  (the  longer  passages  had  more  questions).  The  learner  would  then  proceed  to 
the  next  text  passage  reading  the  text  and  answering  the  questions  at  his  or  her  own  pace.  Simple  navigation  buttons 
along  the  bottom  of  every  screen  allowed  the  learner  to  easily  move  back  and  forth  between  text  and  questions  at 
any  time. 

The  three  alternate  treatments  were  delayed  feedback  (DF),  single-try  immediate  feedback  (STF),  and 
multiple-try  immediate  feedback  (MTF).  Note  that  since  retention  test  memory  of  initial  lesson  responses  is  a 
dependent  variable  in  this  study,  in  order  to  prevent  rehearsal  of  initial  lesson  responses,  the  final  feedback  screen 
for  each  item  in  all  three  treatments  did  not  include  item  distractors  (Sassenrath  & Yonge,  1969;  Sturges,  1969). 

The  STF  treatment  provided  the  correct  response  immediately  after  one  learner  response,  whether  the 
response  was  right  or  wrong.  After  a response,  an  arrow  would  point  to  the  correct  alternative  and  the  learner  was 
told  "Right"  when  correct  and  "No,  here  is  the  answer"  when  wrong.  In  either  case,  the  stem  and  correct  answer 
were  displayed,  the  item  distractors  were  not  shown.  The  MTF  treatment  provided  the  correct  response  immediately 
after  a correct  response  like  STF.  However,  with  an  incorrect  response,  the  learner  was  told  ’’No,  try  again"  and 
continued  to  select  answers  until  the  correct  response  was  selected,  then  an  arrow  would  point  to  the  correct 
response  and  the  learner  was  told  "Right".  Then  the  stem  and  correct  answer  were  displayed,  the  item  distractors 
were  not  shown.  Note  that  STF  and  MTF  are  identical  when  the  learner’s  initial  response  is  correct,  but  obviously 
differ  when  the  learner’s  response  is  incorrect.  The  DF  treatment  required  the  learner  to  respond  to  questions  and 
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move  on  without  any  immediate  feedback.  After  all  text  passage  and  questions  were  completed,  then  all  of  the  items 
were  presented  again  in  the  original  order.  Only  the  correct  responses  were  shown  with  each  question,  the 
distractors  were  not  shown,  and  the  student  could  only  read  the  display.  Thus  the  DF  feedback  screen  displays  were 
identical  to  the  final  feedback  screen  displays  provided  for  STF  and  MTF,  only  the  DF  feedback  screen  displays 
were  presented  after  the  entire  lesson  rather  than  immediately  with  each  item. 

The  purpose  and  requirements  of  the  study  were  explained  in  three  classes  of  students  all  taught  by  the 
same  teacher.  Those  students  choosing  to  participate  collected  consent  forms  to  be  signed  by  a parent  or  guardian. 
About  a week  later,  participants  moved  to  the  school  computer  room  during  their  social  studies  class,  and  were 
randomly  assigned  to  one  of  the  three  computer-based  treatments,  STF,  MTF,  or  DF.  One  day  later,  participants 
completed  the  paper-and-pencil  24-hour  retention  test  in  class. 

The  retention  test  given  a day  after  the  lesson  was  designed  to  measure  memory  of  initial  lesson  responses 
and  of  the  correct  responses.  This  paper-based  retention  test  used  the  same  36  multiple-choice  items  that  were  used 
in  the  computer-based  lesson.  These  36-items  fell  into  two  groups  of  18  items  each,  verbatim  and  inferential.  These 
two  groups  were  further  blocked  into  three  categories  by  lesson  item  difficulty,  easy  (M  = .87),  mid-range  (M  = -72), 
and  difficult  (M  = .50),  with  each  block  containing  six  items.  The  associated  reading  passages  were  not  made 
available  to  the  students  during  retention  testing.  There  were  four  or  five  questions  on  each  8.5-  by  1 1-inch  page. 
Each  question  had  two  blanks,  one  blank  labeled  “1st”  for  the  initial  lesson  response  and  one  blank  labeled  “C”  for 
the  correct  response. 

The  retention  test  contained  the  following  instructions,  “Note  that  each  question  has  two  blanks.  The  first 
blank  is  a check  to  see  if  you  can  remember  the  first  answer  that  you  gave  during  the  computer  lesson.  The  second 
blank  is  for  the  correct  response  to  the  question.”  These  instructions  were  read  aloud  by  the  teacher.  The  teacher 
answered  questions  about  how  to  complete  the  test,  and  then  students  were  given  as  much  time  as  needed  to  finish. 

The  total  amount  of  time  spent  completing  the  lesson  was  also  of  interest.  Lesson  time  data  for  each 
student  was  collected  by  the  computer  program,  and  included  total  time  from  the  display  of  the  first  screen  until  the 
student  exited  the  lesson. 

Results 

Dependent  variables  were  lesson  time,  and  retention  test  recognition  memory  of  initial  lesson  responses  and 
of  correct  responses  with  two  kinds  of  questions  (verbatim  and  inferential)  across  three  levels  of  lesson  item 
difficulty  (difficult,  mid-range,  and  easy).  These  data  were  analyzed  by  separate  analysis  of  variance,  and 
probabilities  were  evaluated  more  conservatively  using  Greenhouse-Geisser  and  Huynh-Feldt  corrections 
automatically  provided  by  the  SYSTAT  8.0  (1998)  analysis  package.  Lesson  time  data  and  retention  test  means  and 
standard  deviations  for  each  treatment  group  at  each  item  kind  and  difficulty  level  are  provided  in  Table  1 . 

Lesson  scores  were  analyzed  as  a check  of  initial  group  equivalence.  The  random  assignment  was  judged 
successful  with  overall  lesson  scores  of  24.4  (of  36  maximum)  for  the  STF  group,  25.4  for  the  MTF  group,  and  25.5 
for  the  DF  group.  The  comparison  of  these  means  using  analysis  of  variance  was  non-significant,  F(l,  51)  = 0.43,  g 
= .65. 

Retention  test  data  were  analyzed  by  a mixed  3 x (2  x 2 x 3)  analysis  of  variance  with  one  between-subjects 
factor,  feedback  condition  (DF,  MTF,  or  STF),  and  three  within-subjects  factors,  type  of  response  (retention  of 
initial  lesson  response  and  of  the  correct  response),  kind  of  question  (verbatim  and  inferential),  and  lesson  item 
difficulty  block  (difficult,  mid-values,  and  easy).  The  interaction  of  feedback  and  type  of  response  was  significant, 
F (2,  49)  = 7.15,  MSE  = 0.023,  p < .01.  Follow-up  Scheffe’  test  showed  that  the  DF  treatment  group  mean  for  ILR 
(M  = 0.84)  was  significantly  larger  than  the  STF  group  mean  for  ILR  (M  = 0.73),  no  other  mean  comparisons  were 
significant. 
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Table  l 

Lesson  time  data  (in  seconds ),  and  lesson  and  retention  test  means  and  standard  deviations  (in  parentheses)  for 
each  treatment  group  for  each  item  kind  and  difficulty  level. 


Lesson  Retention  Test 


Treatment 

Lesson  time 

Item  difficulty 

V 

I 

ILR-V 

ILR-I 

CR-V 

CR-I 

STF  (n=18) 

1577 

Difficult 

0.46 

0.44 

0.66 

0.66 

0.67 

0.83 

(280.6) 

(0.20) 

(0.10) 

(0.18) 

(0.26) 

(0.22) 

(0.18) 

Mid-range 

0.85 

0.57 

0.76 

0.67 

0.79 

0.67 

(0.07) 

(0.13) 

(0.18) 

(0.19) 

(0.18) 

(0.21) 

Easy 

0.92 

0.82 

0.87 

0.78 

0.91 

0.84 

(0.08) 

(0.11) 

(0.12) 

(0.18) 

(0.09) 

(0.12) 

MTF  (n=17) 

1554 

Difficult 

0.51 

0.55 

0.71 

0.67 

0.71 

0.72 

(332.5) 

(0.20) 

(0.16) 

(0.24) 

(0.17) 

(0.20) 

(0.21) 

Mid-range 

0.77 

0.65 

0.79 

0.72 

0.81 

0.74 

(0.07) 

(0.04) 

(0.18) 

(0.16) 

(0.17) 

(0.17) 

Easy 

0.91 

0.83 

0.86 

0.81 

0.91 

0.81 

(0.09) 

(0.09) 

(0.12) 

(0.18) 

(0.12) 

(0.18) 

DF  (n=  1 7) 

1899 

Difficult 

0.48 

0.55 

0.80 

0.81 

0.68 

0.68 

(409.9) 

(0.18) 

(0.14) 

(0.18) 

(0.22) 

(0.25) 

(0.27) 

Mid-range 

0.85 

0.65 

0.84 

0.78 

0.85 

0.78 

(0.06) 

(0.07) 

(0.15) 

(0.18) 

(0.14) 

(0.21) 

Easy 

0.94 

0.78 

0.93 

0.87 

0.89 

0.82 

(0.05) 

(0.08) 

(0.09) 

(0.16) 

(0.10) 

(0.17) 

Note.  Lesson  time  is  in  seconds;  standard  deviations  in  parenthesis;  V = Verbatim  questions,  I = Inferential 
questions;  ILR=  remembers  initial  lesson  response;  CR=  recognize  correct  response;  STF  = Single-try  feedback, 
MTF  = Multiple-try  feedback,  DF  = Delayed  feedback.  Each  item  difficulty  range  is  the  average  of  6 items. 

A significant  effect  was  obtained  for  kind  of  question,  F(l,  49)  = 8.29,  MSE  = 0.035,  p < .01,  indicating 
that  the  verbatim  retention  test  mean  (M  = 0.80)  was  greater  than  the  inferential  retention  test  mean  (M  = 0.76), 
which  simply  reflects  the  lesson  values  for  verbatim  (M  = 0.74)  and  inferential  (M  = 0.65)  questions.  Next,  a 
significant  effect  was  obtained  for  question  difficulty,  F(2,  98)  = 32.20,  MSE  = 0.035,  p < .01.  The  retention  test 
means  for  each  item  difficulty  level  are:  easy  items  (M  = 0.86),  mid-range  difficulty  items  (M  = 0.77),  and  difficult 
items  (M  = 0.71).  As  with  kind  of  question  above,  these  retention  test  values  simply  reflect  lesson  values  which  are: 
easy  items  (M  = 0.87),  mid-range  difficulty  items  (M  = 0.72),  and  difficult  items  (M  = 0.50).  Though  these  findings 
are  significant,  they  have  little  practical  meaning. 

The  three-way  interaction  of  feedback,  type  of  response,  and  question  difficulty  was  significant,  F(4,  98)  = 
3.52,  MSE  = 0.015,  p < .01.  To  further  examine  this  complex  three-way  interaction,  two  separate  follow-up 
ANOVAs  of  retention  test  data  are  reported  below,  one  of  initial  lesson  response  data  and  one  of  correct  response 
data.  But  first  this  three-way  interaction  is  graphically  compared  to  the  delta  rule  predicted  values  (see  Figure  2)  to 
set  the  stage  for  the  follow-up  analyses. 

Since  the  observed  retention  data  shown  in  Figure  2 consists  of  multiple-choice  questions,  these  data  were 
corrected  for  guessing  in  order  to  be  consistent  with  the  predicted  values.  The  correction  for  guessing  formula  from 
Nitko  (1996)  is  corrected  score  = R - W/(n-l)  where  R is  raw  score,  W is  number  wrong,  and  n is  the  number  of 
multiple-choice  alternatives.  The  lines  showing  the  predicted  and  observed  retention  for  initial  lesson  responses 
(left  panel  of  Figure  2)  are  highly  similar  in  both  magnitude  and  form,  suggesting  that  the  connectionist  model 
presented  by  Clariana  (1999)  may  adequately  account  for  24-hour  retention  of  initial  learner  responses.  The  lines 
showing  the  predicted  and  observed  retention  of  correct  responses  (see  right  panel  of  Figure  2)  are  generally  similar 
in  shape  but  are  not  similar  in  magnitude.  Specifically,  the  predicted  correct  response  values  over-estimate  the 
observed  values.  ✓ 
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Figure  2.  Lesson  Response  and  Correct  Response  predicted  retention  test  values  (dashed  lines)  for  immediate 
feedback  (IF)  and  delayed  feedback  (DF)  compared  to  observed  data  corrected  for  guessing  (solid  lines)  for  delayed 
feedback  (DF),  multiple-try  feedback  (MTF),  and  single-try  immediate  feedback  (STF) 

Follow-up  Analysis  of  Retention  Test  ILR  Data 

Retention  test  ILR  data  were  analyzed  by  a mixed  3 x (2  x 3)  analysis  of  variance  with  one  between- 
subjects  factor,  feedback  condition  (DF,  MTF,  or  STF),  and  two  within-subjects  factors,  kind  of  question  (verbatim 
and  inferential)  and  lesson  item  difficulty  (most  difficult,  mid-range,  and  easy).  A significant  effect  was  obtained 
for  feedback,  F(2,  49)  = 4.13,  MSE  = 0.081, p < 05,  as  already  reported  above  that  retention  of  ILRs  was  greater  for 
DF  compared  to  STF  (see  Figure  2),  in  addition  MTF  was  more  like  STF  than  like  DF. 

A significant  effect  was  also  observed  for  kind  of  question  F(2,  49)  = 8.24,  MSE  = 0.024,  p < .01  and  for 
item  difficulty  blocks  F(2,  98)  = 20.24,  MSE  = 0.025,  p < .01.  Retention  test  memory  of  ILRs  for  verbatim 
questions  (M  = 0.80)  was  greater  than  memory  of  inferential  questions  (M  = 0.75).  Retention  test  memory  of  ILRs 
for  difficult  items  (M  = 0.72)  and  for  mid-difficulty  items  (M  = 0.76)  was  less  than  memory  of  easy  items  (M  = 
0.85).  As  above,  though  these  two  findings  are  significant,  they  have  little  practical  meaning. 

Follow-up  Analysis  of  Retention  Test  Correct  Response  Data 

Retention  test  correct  response  data  were  analyzed  by  the  same  mixed  3 x (2  x 3)  analysis  of  variance. 
Three  findings  reached  significance.  A significant  effect  was  obtained  for  item  difficulty  blocks,  F(2,  98)  = 24.74, 
MSE  = 0.025,  p < .01 . Scheffe'  tests  show  that  retention  test  memory  of  correct  responses  for  difficult  items  (M  = 
0.71)  and  for  mid-range  items  (M  = 0.77)  were  both  less  than  memory  of  easy  items  (M  = 0.87),  a finding  of  little 
practical  interest. 

More  importantly,  the  interaction  of  feedback  and  item  difficulty  was  significant,  F(4,  98)  = 2.54,  MSE  = 
0.025,  p < .05  (Greenhouse-Geisser  p = 0.05;  Huynh-Feldt  p = 0.04).  Although  this  disordinal  interaction  was 
directionally  consistent  with  the  delta  rule  predictions,  with  STF  best  for  difficult  items  and  DF  best  for  easy  items 
(see  right  panel  of  Figure  2),  follow-up  Scheffe'  test  obtained  no  significant  differences  for  type  of  feedback  within 
each  question  level.  Thus  the  hypothesized  differences  between  DF  and  STF  at  different  item  difficulty  levels  were 
too  small  to  be  considered  reliable. 

The  interaction  between  kind  of  question  and  lesson  item  difficulty  was  significant,  F(2,  98)  = 7.62,  MSE  = 
0.023,  p < .01.  Though  inferential  lesson  questions  appear  to  be  slightly  more  effective  than  verbatim  questions  for 
difficult  lesson  items,  follow-up  Scheffe'  test  obtained  no  significant  differences  for  type  of  feedback  within  each 
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question  level.  Thus  a possible  instructional  advantage  of  inferential  questions  over  verbatim  questions  (per  Merrill, 
1987)  is  too  small  to  be  considered  reliable. 

Discussion 

The  first  hypothesis  was  confirmed  that  retention  of  initial  lesson  responses  is  greater  for  delayed  feedback 
compared  to  immediate  feedback  across  all  item  difficulties,  but  especially  with  difficult  lesson  items.  The 
association  weights  of  ILR  errors  increased  (see  dashed  line  in  Figure  2).  Further,  the  observed  retention  of  initial 
lesson  responses  for  STF  and  DF  were  very  similar  to  their  corresponding  delta  rule  predicted  values  (see  left  panel 
of  Figure  2).  This  finding  provides  empirical  support  for  the  potential  of  a connectionist  model  to  predict 
instructional  feedback  effects. 

The  practical  value  of  hypothesis  one  for  instructional  design  is  that  in  some  learning  situations,  it  is  critical 
to  remember  initial  lesson  responses,  especially  if  answers  are  not  absolutely  "right"  or  "wrong"  but  serve  as 
learning  transitions  to  broader  understanding.  For  example,  in  discovery  learning  situations,  learners  are  required  to 
remember  and  use  previous  responses.  In  such  cases,  delayed  feedback  or  even  no  feedback  would  allow  learners  to 
maintain  initial  lesson  responses  at  a greater  rate  than  with  immediate  feedback.  On  the  other  hand,  immediate 
feedback  involves  a trade-off  between  increasing  correct  response  associations  at  the  expense  of  forgetting  other 
responses,  and  these  other  responses  are  likely  more  meaningful  to  the  learner  even  though  "incorrect".  In  the  many 
situations  where  it  is  critical  to  strengthen  the  correct  response  and  diminish  the  incorrect  response,  then  immediate 
feedback  would  be  better. 

The  second  hypothesis  that  feedback  has  its  greatest  effect  with  difficult  lesson  items  was  confirmed. 
Lesson  to  retention  test  change  at  each  lesson  item  difficulty  block  expressed  in  effect  sizes  (ES),  calculated  as  the 
difference  between  lesson  and  retention  score  divided  by  the  standard  deviation  of  the  lesson  score,  are:  for  easy 
lesson  items,  ES  = -.06,  for  mid-range  lesson  items,  ES  = .35,  and  for  difficult  lesson  items,  ES  = 1.17.  Previous 
studies  have  provided  the  groundwork  for  this  finding  by  showing  that  feedback  has  its  greatest  effect  with  difficult 
items  (Sturges,  1978).  For  example,  Bangert-Drowns,  Kulik,  Kulik,  and  Morgan  (1991)  state,  "If  feedback’s  primary 
importance  is  the  correction  of  errors,  then  one  would  expect  to  see  larger  effects  for  instruction  with  higher  error 
rates.  This  is  exactly  what  happens"  (p.230).  Thus,  future  feedback  investigations  must  consider  and  control  lesson 
item  difficulty,  or  else  results  may  be  confounded  by  lesson  difficulty. 

The  third  hypothesis,  that  feedback  timing  interacts  with  lesson  item  difficulty,  was  not  supported. 
However  the  mean  differences  were  in  the  right  direction  with  STF  best  with  difficult  items  and  DF  best  with  easier 
items  (see  the  right  panel  of  Figure  2).  This  result  could  be  anticipated  in  that  lesson  items  were  not  difficult  enough 
to  produce  the  interaction,  the  average  lesson  item  difficulty  for  the  most  difficult  question  block  in  this 
investigation  was  M = 0.50.  Note  that  the  delta  rule  predicted  difference  between  immediate  and  delayed  feedback 
would  be  most  pronounced  for  lesson  item  difficulties  less  than  about  0.40  (see  Figure  2),  and  are  actually  predicted 
to  be  identical  for  lesson  item  difficulties  near  0.50.  Thus,  to  adequately  test  this  hypothesis,  an  unusually  difficult 
lesson  would  be  required.  Pragmatically,  for  computer-based  lessons  that  use  multiple-choice  questions  of 
reasonable  difficulty,  immediate  and  delayed  feedback  groups  should  obtain  similar  posttest  scores,  with  perhaps  a 
slight  advantage  for  delayed  feedback. 

Though  the  delta  rule  predicted  ILR  values  well,  this  approach  overestimated  posttest  correct  response 
values.  Using  trend  analysis,  the  values  a ain  = 0.5  and  tG  = 0.85  will  adjust  the  delta  rule  predictions  for  posttest 
correct  responses  to  more  closely  match  the  observed  data.  Future  investigations  should  consider  whether  these 
variables  of  the  delta  rule  are  more  constant  for  classes  of  learner  response,  or  are  more  variable,  perhaps  related  to 
individual  learner  variables. 

What  are  the  effects  of  feedback  immediacy  and  of  multiple  exposures?  MTF  was  much  more  like  STF  in 
retention  test  memory  of  ILRs  indicating  that  feedback  immediacy  acts  to  reduce  memory  of  ILR  errors,  a 
retroactive  interference  effect.  But  MTF  rather  than  clearly  mirroring  STF  (immediacy)  or  DF  (multiple  item 
exposure)  generally  fell  midway  between  STF  and  MTF,  indicating  that  both  feedback  timing  and  number  of 
exposures  combine  or  interact  to  impact  retention  test  memory,  especially  for  memory  of  correct  responses.  This 
combination  or  interaction  of  immediacy  and  multiple-exposures  is  of  theoretical  interest,  and  so  should  be 
addressed  by  additional  experimentation. 

The  findings  of  this  study  involve  only  retention  test  recognition  learning  outcomes  and  should  not  be 
generalized  to  other  types  of  learning  outcomes,  such  as  recall.  Additional  research  should  consider  the  application 
of  a connectionist  model  for  predicting  the  effects  of  feedback  on  higher-level  learning  outcomes. 

As  a footnote,  Kulhavy  and  Stock  (1989)  have  described  an  information  processing  explanation  of 
feedback  effects  based  on  servocontrol  theory  (which  describes  the  interaction  between  system  output,  sensors, 
feedback  from  the  sensors,  and  mechanical  devices  that  impact  output).  Their  model  views  learner's  response 
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confidence  as  a metacognitivc  component  that  controls  or  at  least  strongly  influences  how  feedback  information  is 
processed  by  the  learner.  So  far  (since  Sturges,  1978),  response  confidence  studies  have  shown  mixed  results 
(Bangert-Drowns,  Kulik,  Kulik,  & Morgan,  1991;  Mory,  1992;  1994),  so  whether  response  confidence  serves  a 
metacognitive  function  in  feedback  processing  is  unknown.  Requiring  a learner  to  consciously  consider  the 
confidence  of  every  lesson  response  would  obviously  alter  the  expected  and  normal  pattern  of  a lesson,  for  example 
increasing  the  amount  of  time  the  learner  takes  with  each  item.  Further,  asking  the  learner  for  response  confidence 
at  each  lesson  response  may  be  distracting,  and  could  disrupt  the  learning  process.  Also,  it  has  been  suggested  by 
Mory  (1994)  that  learners'  self-report  of  response  confidence  is  inaccurate  in  some  cases  (see  also  Metcalfe,  1986). 
For  these  reasons,  response  confidence  was  not  used  in  this  present  investigation.  However,  response  confidence 
could  serve  as  a logical  alternative  measure  of  initial  lesson  output  activation  level,  a^,  in  the  delta  rule  calculations. 
Thus,  future  investigation  of  the  possible  metacognitive  effects  of  response  confidence  may  obtain  added  insight  by 
applying  a connectionist  model. 

(Note:  A more  detailed  version  of  this  manuscript  has  been  accepted  for  publication  in  ETR&D.  Special  thanks  to 
Dave  Jonassen  for  his  assistance  in  forming  the  idea  for  the  study  and  for  directing  several  doctoral  students  my 
way,  and  also  thanks  to  Steve  Ross  and  the  ETR&D  reviewers  for  their  suggestions  and  recommendation  for  the 
manuscript.) 
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Abstract 

The  purpose  of  this  study  was  to  examine  the  effects  that  level  of  field  dependence  and  differentially  coded 
instructional  materials  (black  and  white  and  colored)  had  on  female  student  achievement  of  different  educational 
objectives.  One  hundred  twenty  six  college  level  students  participated  in  the  study.  On  the  Total  criterion  measure 
significant  achievement  differences  occurred  in  favor  of  Field  Independent  students.  A non  significant  interaction 
between  color-coding  and  field  dependence  was  found  on  the  Total  Test  scores.  Significant  differences  in 
achievement  were  also  found  in  favor  of  females  who  received  the  color-coded  version  of  the  Total  Test.  These 
results  confirm  previous  results  relating  to  the  importance  of  field  dependence  in  learning  situations  and  stimulates 
the  necessity  for  further  more  sophisticated  research  to  systematically  analyze  the  unique  contributions  that  color- 
coded  instructional  strategies  might  have  in  facilitating  the  achievement  of  female  students. 

Introduction 

A great  deal  of  interest  has  focused  on  the  concept  of  learning  style  as  it  relates  to  the  way  in  which 
individuals  process  information  (Bjorklund,  1989).  The  learning  style  concept  grew  out  of  the  research  on  how 
people  perceive,  interact  with  and  organize  information  received  from  the  world  around  them.  Results  from  similar 
research  suggest  that  individuals  differ  in  how  they  approach  a learning  task  (Moore,  1985;  Moore  & Bedient,  1986; 
Reardon  & Moore,  1988)  but  that  these  variations  do  not  reflect  levels  of  intelligence  (Woolfolk,  1993).  Instead 
they  have  to  do  with  the  “...preferred  ways  that  individuals  have  for  processing  and  organizing  information  and  for 
responding  to  environmental  stimuli”  (Shuell,  1981,  p.  46).  Probably  the  most  extensively  researched  cognitive 
style,  which  has  the  widest  application  to  educational  problems,  is  known  as  field  dependence/independence 
(Witkin,  et  al.,  1962).  The  field  dependence/independence  continuum,  as  it  applies  to  learners,  describes  the  degree 
to  which  learners  will  interact  with  a visual  presentation,  that  is,  whether  the  learner  will  merely  interact  with  the 
visual  as  presented  or  will  he/she  analyze,  reorganize  and  synthesize  the  instructional  field  to  make  the  content  more 
meaningful  and  memorable  (Jonassen  & Grabowski,  1993).  Field  dependent  individuals,  when  presented  a 
visualized  presentation  tend  to  modify  the  structure  but  accept  and  interact  with  it  as  it  is  presented.  They  tend  to 
fuse  all  segments  within  the  visual  field  and  do  not  view  or  interact  with  the  visual  components  discretely.  Field 
independents  tend  to  act  upon  a visual  stimulus,  analyzing  it  when  it  is  organized  and  providing  their  own  structure 
when  it  lacks  organization.  The  findings  of  most  studies  in  the  literature  examining  achievement  indicate  a 
superiority  of  FID  students  over  FD  student  in  terms  of  performance  (Canelos,  Taylor,  & Gates,  1980;  Witkin,  et  al., 
1977). 

Although  many  studies  have  examined  the  effects  of  visual  attributes  on  learning  (Dwyer,  1978,  1987)  few 
have  studied  the  effects  of  varied  visual  attributes  on  specific  cognitive  learning  styles.  Research  has  shown  that 
color-coding  helps  learners  organize  or  categorize  information  into  useful  patterns,  which  enables  them  to  interpret 
and  adjust  more  readily  to  their  environment.  It  was  hypothesized  that  color-coded  visuals  would  be  more  effective 
than  black  and  white-coded  visuals  in  enhancing  the  solvent  visual  cues,  thereby  making  them  more  identifiable  and 
instructional  to  field  dependent  learners.  The  color-coding  would  attempt  to  compensate  for  the  restructuring  skills 
absent  in  the  field  dependent  learners  and  subsequently  lead  to  deeper  information  processing  and  increased 
achievement.  This  hypothesis  seemed  plausible  since  field  dependent  learners  tend  to  be  global  in  perception  and 
would  be  most  inclined  to  take  advantage  of  the  increased  structure  provided  by  the  color-coding.  In  a study  on 
gender  preference  in  imagery,  Rogers  (1995b)  found  that  “girls  like  colors,  boys  like  action.”  Likewise,  Freedman 
(1989)  reported  when  using  computer  graphics  fifth-  grade  girls  were  more  concerned  than  boys  with  using  color 
and  color  combinations.  Previous  research  has  verified  the  superiority  (Dwyer,  1978,  1987)  of  color-coded  visuals 
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over  black  and  white  illustrations  in  facilitating  achievement  of  specific  types  of  learning  objectives.  However, 
although  it  has  been  found  that  males  are  generally  more  field  independent  than  females  (Witkin,  et  al.,  1971)  it  was 
not  known  whether  color-coding  (B&W  and  Color)  and  field  dependence  interact  or  whether  color-coding  might 
positively  effect  the  field  dependency  of  females  by  enabling  them  to  more  effectively  and  efficiently  organize  and 
structure  instructional  content. 

Statement  of  the  Problem 

Specifically,  the  purpose  of  this  study  was  to  examine  the  effect  that  color-coding  (B&W  and  Color)  has  on 
the  achievement  different  educational  objectives  of  female  students  categorized  as  field  dependent  or  field 
dependent. 

Method 

One  hundred  twenty  six  female  students  enrolled  in  basic  education  courses  at  The  Pennsylvania  State 
University  and  Virginia  Tech  University  participated  in  this  study.  Students  were  classified  as  field  dependent,  field 
neutral,  or  field  independent  as  a result  of  their  performance  on  the  Group  Embedded  Figures  Test  (GEFT),  (Witkin, 
Oltman,  Raskin,  & Karp,  1971)  and  were  divided  into  the  different  levels  based  on  their  mean  achievement  level  on 
the  GEFT. 

Each  student  in  each  treatment  received  the  Group  Embedded  Figures  Test  (GEFT)  (Witkin,  et  al.  1971). 
The  GEFT  is  a group-administered,  25-item  test  administered  in  three  timed  sections  (2,  2,  and  5 minutes  each). 
Students  must  trace  one  of  eight  simple  figures  embedded  in  figures  of  greater  complexity.  Reported  reliability  is 
.82.  Students  participating  in  these  studies  were  classified  as  field  independent  (FI),  field  neutral  (FN)  in  field 
dependent  (FD)  based  on  their  performance  on  the  GEFT.  The  grand  mean  for  this  study  was  13.22  with  a standard 
deviation  of  4.17.  Students  achieving  one-half  standard  deviation  above  the  grand  mean  were  considered  to  be  FI 
while  students  achieving  one-half  standard  below  the  mean  were  considered  to  be  FD.  Field  neutral  students  were 
those  achieving  one-half  standard  deviation  on  either  side  of  the  mean.  The  range  of  scores  achieved  on  the  GEFT 
was  17. 

The  subject  content  for  the  studies  consisted  of  a 2,000  word  instructional  booklet  on  the  anatomy  and 
functions  of  the  human  heart.  Each  booklet  contained  nineteen  illustrations,  which  were  designed  to  illustrate  the 
content.  The  illustrations  in  the  black  and  white  version  contained  black  and  white  coded  line  drawings,  which 
highlighted  the  information  and  process  being  presented.  Students  receiving  the  color-coded  treatment  received  the 
same  visuals  as  did  students  receiving  the  black  and  white  treatment;  however,  several  different  colors  were  used  to 
highlight  the  information  being  discussed.  After  receiving  their  respective  coded  treatments  each  student  then 
received  a battery  of  four  individual  tests  (drawing,  identification,  terminology,  and  comprehension)  and  were  then 
were  combined  into  the  80-item  Total  criterion  score.  These  tests  were  designed  to  measure  achievement  of 
different  types  of  educational  objectives.  The  major  independent  variables  in  the  study  were  the  effect  that  B&W 
and  Color-Coding  of  information  had  on  the  information  processing  strategies  of  male  and  female  students  identified 
as  processing  different  levels  of  field  dependence  (FI,  FN,  FD). 

Criterion  Measures 

The  following  description  of  the  Criterion  Tests,  adapted  from  Dwyer  (1978,  pp.  45-47)  illustrates  the  types 
of  instructional  objectives  assessed  in  this  study,  summarized  into  the  Total  Criterion  Score. 

Drawing  Test. 

The  objective  of  the  drawing  test  was  to  evaluate  student  ability  to  construct  and/or  reproduce  items  in  their 
appropriate  context.  The  drawing  test  provided  the  students  with  a numbered  list  of  terms  corresponding  to  the  parts 
of  the  heart  discussed  in  the  instructional  presentation.  The  students  were  required  to  draw  a representative  diagram 
of  the  heart  and  place  the  numbers  of  the  listed  parts  in  their  respective  positions.  For  this  test,  the  emphasis  was  on 
the  correct  positioning  of  the  verbal  symbols  with  respect  to  one  another  and  in  respect  to  their  concrete  referents. 
Identification  Test. 

The  objective  of  the  identification  test  was  to  evaluate  student  ability  to  identify  parts  or  positions  of  an 
object.  This  multiple-choice  test  required  students  to  identify  the  numbered  parts  on  a detailed  drawing  of  a heart. 
Each  part  of  the  heart,  which  had  been  discussed  in  the  presentation,  was  numbered  on  a drawing.  The  objective  of 
this  test  was  to  measure  the  ability  of  the  student  to  use  visual  cues  to  discriminate  one  structure  of  the  heart  from 
another  and  to  associate  specific  parts  of  the  heart  with  their  proper  names. 
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Terminology  Test 

This  test  consisted  of  items  designed  to  measure  knowledge  of  specific  facts,  terms,  and  definitions.  The 
objectives  measured  by  this  type  of  test  are  appropriate  to  all  content  areas  that  have  an  understanding  of  the  basic 
elements  as  a prerequisite  to  the  learning  of  concepts,  rules,  and  principles. 

Comprehension  Test 

Given  the  location  of  certain  parts  of  the  heart  at  a particular  moment  of  its  functioning,  the  student  was 
asked  to  determine  the  position  of  other  specified  parts  or  positions  of  other  specified  parts  of  the  heart  at  the  same 
time.  This  test  required  that  the  students  have  a thorough  understanding  of  the  heart,  its  parts,  its  internal 
functioning,  and  the  simultaneous  processes  occurring  during  the  systolic  and  diastolic  phases.  The  comprehension 
test  was  designed  to  measure  a type  of  understanding  in  which  the  individual  can  use  the  information  being  received 
to  explain  some  other  phenomenon. 

Total  Criterion  Score 

The  items  contained  in  the  above  noted  individual  Criterion  Tests  were  combined  into  a composite  test 
score  (Total  Criterion  Test).  The  purpose  was  to  measure  total  achievement  of  the  objectives  presented  in  the 
instructional  unit.  This  was  the  unit  of  data  used  in  this  study  for  analysis. 

Analysis 

A 3x2  analysis  of  variance  was  used  to  analyze  student  achievement  in  the  Total  Criterion  Test.  The  range 
of  scores  possible  on  this  test  was  0-80.  Main  effects  considered  were  cognitive  style  (field  dependent  (FD),  neutral 
(N),  and  field  independent  (FI)),  color-coding  (color  and  black  & white)  and  gender.  Interactions  among  cognitive 
style,  color-coding  and  gender  were  also  of  interest.  The  alpha  level  was  set  at  .05  for  the  analysis.  The  Kuder- 
Richardson  Formula  20  Reliability  coefficient  of  .92  was  computed  for  the  Total  Criterion  Test  from  a random 
sampling  of  prior  studies  (Dwyer,  1978,  p 47). 

Results  and  Discussion 

The  results  achieved  on  the  Total  Test  supports  the  expected  contention  that  field  independent  learners  (M= 
60.5 1_)  were  superior  to  field  dependent  learners  (M=  45.20)  in  the  cognitive  processes  they  in  processing 
information  related  to  different  types  of  learning  objectives  use  (F  (2,)  = 12.14,  p.  < .05).  See  Table  1 for  means  and 
standard  deviations. 

There  was  a significant  main  effect  on  color-coding  (F  (1,1 22)=8.92,  p.  < .05),  the  analysis  of  variance 
conducted  on  performance  data  achieved  by  female  students  on  the  Total  Criterion  Test.  The  analysis  indicated  that 
females  who  received  the  color-coded  treatments  (M=  57.95)  achieved  significantly  higher  scores  on  the  Total  than 
did  female  students  who  received  the  black  and  white  coded  treatments  (M=  49.61).  The  analysis  further  indicated 
that  no  significant  interaction  existed  (F  (1,122)  = 1.77,  p.  >.  05)  between  field  dependence  and  color-coding.  See 
Table  1 for  means  and  standard  deviations. 


Table  L Means  and  Standard  deviations.  Field  Dependence 


Effect 

Count 

Mean 

Std.  Dev. 

FD 

45 

60.51 

11.28 

FN 

49 

53.55 

14.52 

FD 

34 

45.21 

15.92 

Table  2.  Means  and  standard  deviations.  Color  and  Black  and  White 


Effect 

Count 

Mean 

Std.  Dev. 

Color 

64 

57.95 

14.09 

B&W 

64 

49.61 

14.85 
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Discussion 

The  results  of  this  experimental  study  support  the  contention  that  field  independent  and  field  dependent 
learners  differ  in  the  cognitive  processes  they  use  as  in  the  effectiveness  of  these  cognitive  processes  as  measured  on 
tests  measuring  different  educational  objectives.  This  study  found  where  female  FI  and  FD  students  reviewed  either 
black  and  white  coded  or  color  -coded  treatments,  the  field  independent  students  scored  significantly  higher  than  did 
field  dependent  students  on  the  Total  Criterion  Test.  These  results  support  prior  research  (Moore,  1986)  which 
found  that  field  independent  learners  tend  to  score  higher  on  criterion  measures  which  require  the  acquisition  of 
information  from  visualized  instruction  and  are  used  to  assessing  visually  complemented  instruction.  This  finding  is 
also  consistent  with  the  previous  reviews  of  the  literature  that  have  concluded  that  field  independent  learners  exhibit 
an  active,  hypothesis  testing  strategy  toward  learning,  whereas  field  dependent  learners  tend  to  employ  a more 
tentative  or  spectator  approval  to  learning  (Witkin  et  al,  1971).  The  results  also  indicated  that  the  use  of  color- 
coding of  visualization  did  not  provide  sufficient  structuring  of  the  critical  information  to  alter  the  information 
processing  level  of  field  dependent  learners. 

It  was  expected  that  the  color-coded  treatments  would  make  the  relevant  cues  more  explicit,  thereby 
improving  their  performance.  The  results  support  this  hypothesis.  Students  who  viewed  the  color-coded  materials 
scored  significantly  higher  than  did  students  viewing  the  black  and  white  materials,  thus  supporting  the  argument 
that  color-coded  materials  can  be  of  instructional  support  at  least  to  female  students. 

These  findings  indicating  that  color-coded  materials  to  be  more  effective  for  females  than  the  black  and 
white  coded  materials  are  among  the  first  findings  which  lend  support  to  the  possibility  of  differential  information 
processing  within  the  gender  domain.  It  leads  to  the  hypothesis  that  the  color-coded  materials  may  have  instigated 
an  active  hypothesis  testing  strategy  toward  learning,  where  the  students  utilized  the  inherent  color-coding  strategy 
to  organize  and  structure  the  information  being  presented. 

Additionally,  the  results  may  have  been  reinforced  if  the  students  were  made  aware  of  the  intent  of  the 
coding  structure  and/or  were  given  a orientation  session  as  to  how  they  might  employ  the  coding  strategies  to 
improve  their  information  acquisition.  Under  these  circumstances  the  color-coded  treatments  might  have  been  even 
more  effective  in  communicating  the  explicit  learning  criteria  to  the  learners. 
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Abstract 

Learning  organizations  are  discussed  within  business  and  industrial  frameworks,  yet  educational 
enterprises  have  sought  similar  revitalization.  Today's  K-12  and  university  systems  grapple  with  change, 
particularly  that  of  telecommunications  technology.  This  session  reports  on  several  Midwest  US  initiatives  to  foster 
learning  organizations.  These  projects  demand  greater  inter-institutional  collaboration  as  leaders  purposefully 
create  learning  networks. 

Introduction 

Authors  like  Peter  Senge,  Chris  Argyris,  Victoria  Marsick,  and  Karen  Watkins  have  been  conducting 
research  about  learning  organizations  for  the  past  several  decades.  While  their  published  principles  make  sense  for 
leaders  in  business  and  industrial  settings  --  the  organizations  within  which  they  conducted  their  studies  — these 
same  policies  and  practices  must  be  translated  appropriately  into  educational  settings.  This  session  presents  the 
journey  of  several  educational  initiatives  that  have  embraced  organizational  change,  intentionally  working  to 
develop  individual  educator,  school,  district,  consortium,  and  statewide  capacity  with  technology  within  school  and 
university  settings.  The  presenters,  who  work  in  different  educational  institutions  and  levels  but  who  collaborate  at 
the  State  level  on  one  of  these  projects,  will  explain  the  implementation  of  concepts  about  learning  organizations 
into  a K-12  and  teacher  education  environment,  especially  involving  instructional  technology. 

Dimensions  of  a Learning  Organization 

A learning  organization  has  been  defined  as  one  that  is  "continually  expanding  its  capacity  to  create  its 
future"  (Senge,  1990,  p.  14).  Such  enterprises  require  continuous  learning  at  all  organizational  levels,  plus 
procedures  to  share  new  knowledge  so  that  the  organization  improves  (Marsick  & Watkins,  1999).  Within 
organizations,  learning  takes  place  at  the  individual,  team,  and  institutional  levels  with  connections  made  outside  to 
professional  associations,  communities,  and  the  global  environment.  Learning  also  takes  place  through  various 
methods:  formal  education,  training  and  staff  development,  informal  learning,  and  incidental  learning  (Marsick  & 
Watkins,  1990).  Features  within  the  organizational  environment,  such  as  incentives,  communications,  and 
information  technology  effect  the  capability  of  an  organization  to  learn  too. 

Leadership  Policies  for  Learning  Organizations 

Policy  development  toward  creating  and  sustaining  learning  organizations  by  visionary  leaders  provides  the 
greatest  impact.  These  leaders  not  only  share  their  own  conceptions  of  organizational  possibilities,  they  encourage 
others  to  contribute  to  that  vision,  and  they  follow  through  with  implementation  of  policy  and  procedures.  Although 
leaders  may  be  found  at  the  top,  a shared  vision  involves  leaders  and  advocates  of  systemic  improvement  at  all 
levels  of  the  organization.  Such  leaders  attend  to  the  changes  happening  outside  and  within  the  organization;  they 
seek  to  maximize  the  flow  and  sharing  of  information  by  creating  knowledge  management  systems;  and  they 
promote  learning  throughout  the  organization  by  setting  priorities,  encouraging  self-directed  learning,  and  noting  the 
individual  and  group  contributions  through  incentives  and  recognition  (Marsick  & Watkins,  1999). 

Leaders  must  foster  a supportive  environment  of  trust  and  shared  responsibilities  without  fault-finding, 
where  tacit  assumptions  can  be  challenged  and  double-looped  learning  can  take  place  (Argyris,  1999).  Senge,  et  al 
(1999)  provide  practical  suggestions  for  policy  at  each  stage  of  the  progression  of  a learning  organization's 
development.  When  initiating  a learning  organization,  leaders  make  time  for  innovation,  gamer  support,  show  the 
relevance  of  the  initiative,  and  personally  exemplify  the  changed  attitudes  and  behaviors  of  the  five  disciplines.  To 
sustain  the  transformation  once  a learning  organization  has  been  successfully  launched,  leaders  seek  to  overcome 
fear  and  anxiety,  find  ways  to  accurately  measure  impact  of  change  initiatives,  and  integrate  the  innovation  within 
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the  organization  ~ unifying  divisions  that  often  emerge.  As  a learning  organization  becomes  fully  established, 
Senge,  et  al  challenge  leaders  to  diffuse  innovative  change  throughout  the  organization  as  governance,  strategies, 
and  purposes  are  re-examined  and  modified. 

The  school  is  not  now  a learning  organization  as  Michael  Fullan  and  other  writers  remind  us.  Rather, 
characteristics  such  as  irregular  waves  of  change,  episodic  projects,  fragmentation  of  effort,  and  grinding  overload  is 
the  lot  of  most  schools.  The  vast  majority  of  change  efforts  are  misconceived  because  they  fail  to  understand  and 
harness  the  combined  forces  of  moral  purpose  and  skilled  change  agentry.  The  challenge  for  all  educators  is  to  work 
toward  greater  understanding  of  the  new  work  of  principals  and  teachers  as  they  transform  their  schools  from 
bureaucratic  organizations  to  thriving  communities  of  learners  (Fullan,  1998).  Recent  education  publications  and 
initiatives  tell  the  story  of  increased  momentum  among  education  entities  to  embrace  concepts  of  learning 
organizations. 

Impact  of  Technology  on  Educational  Enterprises 

Perhaps  no  innovation  of  the  past  century  has  pervaded  our  society  like  the  telecommunications  technology 
revolution.  Within  the  past  five  years  the  World  Wide  Web  has  been  commercialized  with  a global  business 
economy  emerging.  Similar  infusion  of  telecommunications  is  radically  changing  the  face  of  education.  For 
example,  one  third  of  higher  education  institutions  in  the  United  States  offered  distance  education  courses  in  1998; 
roughly  60  percent  of  these  were  delivered  via  the  Internet  (U.S.  Department  of  Education,  1999).  Public  schools 
throughout  the  country  are  being  equipped  with  computer  labs,  networked  classroom  computers,  and  interactive 
television  technologies.  Throughout  the  country  various  projects  and  centers  are  funded  to  integrate  these 
technologies  appropriately  into  the  curriculum.  The  South  Dakota  stories  and  projects  shared  in  the  work  of  this 
study  support  the  momentum  of  school  change  through  technology. 

The  Learning  Organizations  for  Technology  Integration  (LOFTI)  Project 

One  exemplary  project  is  the  Learning  Organizations  for  Technology  Integration  (LOFTI)  Project,  a five- 
year  major  grant  to  the  State  of  South  Dakota  from  the  U.S.  Department  of  Education  for  staff  development  with 
technology.  Begun  in  1998,  the  LOFTI  Project  aims  to  provide  educators  with  the  knowledge,  skills,  and  abilities 
essential  for  teaching  and  learning  in  a changing  technological  world.  Technology  requires  a great  deal  of 
collaboration  among  organizational  players,  and  it  is  the  catalyst  for  systemic  change  within  smaller  projects 
undertaken  by  the  various  educational  organizational  players.  Thus  the  LOFTI  Project  networks  educators  at  all 
levels  --  K-12,  distance  consortia,  university,  and  teacher  education  — to  share  resources  and  learning  about 
technology. 

The  LOFTI  Project  has  organized  summer  academies,  known  as  "Technology  for  Teaching  and  Learning  " 
(TTLs)  to  develop  and  improve  individual  educator  knowledge,  skills,  and  abilities  within  specialized  roles. 
Hundreds  of  teachers,  administrators,  and  technology  coordinators  are  participating  in  one  to  three  week-long, 
intensive  TTLs  for  advanced  technology  skills,  networking,  technology  for  administrators,  or  distance  learning 
technology.  The  Project  is  unique  because  it  seeks  to  intentionally  incorporate  the  five  disciplines  of  learning 
organizations  (Senge,  1990)  into  its  goals  of  statewide  technology  integration  within  education. 

One  subgroup  of  the  project  consists  of  four  showcase  schools  who  have  been  selected  to  integrate 
technology  completely  into  the  educational  process  as  a model  to  others.  University  players,  another  subgroup, 
direct  their  efforts  primarily  toward  faculty  improvement  and  curriculum  development  in  teacher  education  using 
technology.  Similarly,  the  LOFTI  Project  funds  five  K-12  consortia  that  had  been  set  up  within  the  State  for 
distance  education  to  expand  their  services  and  improve  participating  teachers  in  developing  technology  skills.  The 
South  Dakota  Board  of  Regents  and  the  State  Department  of  Education  and  Cultural  Affairs  have  teamed  up  to 
provide  a 15  hour  telemediated  inservice  curriculum  for  technology  using  the  State’s  newly  formed  Electronic 
University  Consortium  (see  below).  The  Project  is  led  by  the  Technology  & Innovations  in  Education  (TIE)  Office 
in  Rapid  City.  TIE  coordinates  several  leadership  groups  for  the  project,  such  as:  (1)  the  Design  Team  which 
handles  the  day  to  day  operations  of  LOFTI;  (2)  the  Steering  Committee,  made  of  up  representatives  of  each 
organizational  player,  it  defines  policies  and  procedures;  (3)  the  Evaluation  Team,  led  by  an  external  evaluator  from 
Maine,  this  group  measures  the  project’s  success  in  meeting  its  goals  and  objectives;  and  (4)  the  Awareness, 
Advocacy,  and  Advisement  group  is  made  up  of  strategic  State  government,  business,  and  educational  partners  who 
provide  input  in  LOFTI  Project  direction  and  promote  it  to  other  groups. 
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Discover  South  Dakota 

Discover  South  Dakota  is  a multi-dimensional  telecollaborative  curriculum  and  professional  development 
initiative.  The  three  major  strands  of  work  within  the  project  focus  on  the  design  and  implementation  of  (l)student 
telecollaborative  curriculum,  (2)  a professional  development  model  for  Discover  South  Dakota  teachers,  and  (3)  a 
professional  development  model  for  school  principals. 

Students  participating  in  Discover  South  Dakota  are  organized  in  both  classroom-based  learning 
communities  and  on-line  learning  communities  where  they  engage  in  tasks  constructed  around  identified  academic 
content  standards,  technology  standards,  and  information  literacy  standards.  The  curriculum  is  framed  in  the  study 
of  South  Dakota  with  an  integrated,  project-based  design  for  learning.  Students  conduct  research,  communicate  and 
collaborate  with  experts  and  students  across  the  state,  and  publish  to  the  Discover  South  Dakota  web  site.  Students 
make  use  of  e-mail,  threaded  discussion,  and  chat  features  of  the  telecommunications  environment  while  conducting 
their  work.  Partnerships  with  other  education  related  entities  throughout  South  Dakota,  including  university  pre- 
service students  and  teachers,  South  Dakota  Cultural  Heritage  Center  programs,  and  a variety  of  experts,  are  critical 
to  Discover  South  Dakota  curriculum  work. 

During  the  1999-2000  year,  fourth,  fifth,  and  sixth  grade  students  in  210  classrooms  across  the  state 
participated  in  the  Discover  South  Dakota  curriculum  experience. 

A project-embedded  approach  has  emerged  as  a professional  development  model  for  teachers 
implementing  Discover  South  Dakota.  This  model  consists  of  a web  of  professional  development  connections 
available  through  multiple  delivery  systems.  It  includes  combinations  of  face-to-  face  and  on-line  experiences  that 
take  advantage  of  both  synchronous  and  asynchronous  features  of  telecommunications.  These  are  supported  with 
print  materials,  site  visits,  and  technical  assistance.  Through  evaluation  of  the  design  and  implementation  of 
components  of  this  professional  development  model,  the  project  seeks  to  further  our  understanding  of  powerful 
combinations  for  maximizing  the  success  of  teachers  implementing  telecollaborative  teaching  and  learning  in 
classrooms.  In  particular,  data  to  study  inhibitors  and  facilitators  to  this  model  are  being  collected. 

The  Discover  South  Dakota  leadership  strand  is  built  on  the  premise  that  school  leaders  are  key  to 
successful  implementation  of  innovations  in  schools.  The  work  for  school  leaders  is  organized  around  a problem- 
based  learning  framework  focused  on  project-embedded,  just  in  time  delivery  of  professional  development.  The 
overarching  problem  positions  Discover  South  Dakota  as  a technological  innovation  being  implemented  in  their 
respective  schools.  In  order  to  examine  their  current  practices,  participants  engage  in  a series  of  topics  and  activities 
designed  to  correlate  with  the  activities  of  teachers  and  students  during  the  ten  week  implementation  of  Discover 
South  Dakota.  This  component  was  conducted  as  a pilot  during  the  1999-2000  year  and  will  be  scaled  up  during 
2000-2001.  Data  are  collected  to  study  connections  between  successful  implementation  of  Discover  South  Dakota 
by  teachers  when  school  leaders  are  participating  in  this  companion  professional  development  experience. 

Connecting  the  Schools 

The  Connecting  the  Schools  (CTS)  project  is  the  follow-up  endeavor  to  Governor  Bill  Janklow's  Wiring  the 
Schools  project  (wts.state.sd.us).  Wiring  the  Schools  established  a solid  local  area  network  (LAN)  and  electrical 
infrastructure  in  the  K-12  school  buildings  across  South  Dakota.  CTS  is  building  on  that  foundation  for  the  creation 
of  a statewide  video  and  data  intranet  to  improve  the  educational  opportunities  for  K-12  students  in  South  Dakota. 
CTS  has  3 distinct  phases: 

1 . Equipment  seeding 

2.  Network  infrastructure 

3.  Distance  learning 

The  equipment  seeding  is  a one-time  distribution  of  hardware  and  software  to  insure  that  schools  have  a 
quality  LAN  infrastructure  to  build  upon.  Many  schools  will  go  from  having  little  or  no  technology  to  a level  that 
will  allow  them  to  quickly  leap  into  the  21st  century!  The  equipment  phase  began  in  May  1999  and  will  conclude  in 
September  1999.  The  network  infrastructure  named  Digital  Dakota  Network  (DDN)  provides  a frame  relay  or  ATM 
T1  to  approximately  400  public  school  buildings  in  the  state.  The  general  rule  is  that  elementary  schools  (K-6) 
receive  frame  relay  and  that  grades  7-12  receive  ATM.  The  frame  relay  circuits  deliver  data  communications  (e.g., 
Internet,  World  Wide  Web,  e-mail)  and  the  ATM  circuits  deliver  data  and  video.  Telecommunications  companies  in 
the  State  are  installing  interactive  video  room  systems  in  the  distance  learning  classrooms. 

E-mail  and  web  hosting  services  are  also  being  provided.  Dakota  State  University  is  providing  these 
services  under  contract  to  the  CTS  project.  Thin  client  technology  will  serve  Exchange  collaboration  tools  to 
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teachers  and  administrators  across  the  state.  Web  hosting  will  allow  the  publishing  of  district,  school,  teacher, 
classroom  and  student  information. 

The  distance  learning  phase  of  the  project  will  remove  limitations  for  students  and  provide  them  learning 
opportunities  regardless  of  geographic  location.  The  video  conferencing  service  will  provide  two-way  audio  and 
video  communications  to  an  estimated  200  schools  across  the  state.  The  distance  learning  phase  is  going  to  take 
significant  cooperation  and  planning  amongst  South  Dakota  schools.  The  scheduling,  curriculum  sharing,  and 
communicating  of  available  resources  is  absolutely  vital  to  the  success  of  this  phase.  The  distance  learning  services 
will  be  available  in  September  of  2000. 

Electronic  University  Consortium 

Still  in  its  planning  stages,  the  Electronic  University  Consortium  (EUC)  seeks  to  coordinate  the  efforts  of 
South  Dakota's  six  regential  institutions  toward  off-campus  and  distance  provision  of  courses  and  programs.  It  will 
establish  a single  Web  point  of  contact  for  students  to  access  a variety  of  services  --  from  admissions,  registration, 
billing,  and  transcript  services  through  the  delivery  of  courses  and  library  services.  The  EUC  will  coordinate  the 
curriculum  development  effort  of  the  various  institutional  members  while  avoiding  unnecessary  duplication.  In  new 
curricular  development,  its  focus  will  be  on  meeting  the  information  technology  needs  of  business,  industry, 
medicine,  government,  and  K-12  enterprises.  Its  outreach  will  be  to  those  who  cannot  conveniently  attend  classes 
on  a regular  college  campus.  A fifteen  credit  hour  telemediated  teacher  education  curriculum  developed  through  the 
LOFTI  Project  provides  an  impetus  for  other  EUC  curricular  development. 

Observations  and  Reflections 

This  presentation  information  summary  concludes  with  some  of  our  reflections  about  the  successes, 
growth,  and  challenges  provided  by  the  technology  initiatives  and  efforts  towards  organizational  learning  reported  in 
this  paper.  Although  we  have  been  involved  in  variety  of  organizations,  this  is  the  first  time  we  have  seen  the  major 
educational  players  in  the  State  come  together  ostensibly  for  the  purpose  of  collaboration,  teamwork,  and  shaping  its 
future  capacity  building  through  creating  learning  organizations  with  technology.  Several  times  yearly,  Statewide 
meetings  focus  on  various  aspects  of  building  learning  organizations,  and  those  in  local  leadership  positions  of  the 
LOFTI  Project  are  uniquely  placed  to  facilitate  the  greater  understanding  and  buy-in  to  these  concepts.  Many  of  the 
innovations  with  technology  overlap  with  LOFTI,  such  as  the  Discover  South  Dakota,  Connecting  the  Schools,  and 
the  Electronic  University  Consortium,  giving  educators  first-hand  experience  with  collaboration  and  initiative 
development  with  technology. 

The  positive  aspects  of  learning  organization  building  are  pronounced.  These  projects  are  benefited  by 
visionary  leaders  who  make  the  building  of  learning  organizations  a top  priority.  Besides  the  practical  day-to-day 
projects  undertaken  by  various  project  members,  these  initiatives  facilitate  greater  discussion  about  technology  and 
the  future  that  it  plays  in  the  educational  future  of  the  State.  Slowly  but  surely  a vision  of  exciting  possibilities  is 
emerging  where  South  Dakota  educational  enterprises  are  not  only  about  to  better  serve  their  citizens  through 
technology,  but  they  are  also  able  to  reach  out  nationally  and  globally  to  become  a greater  player  in  the  global 
telecommunciations  economies  of  the  future. 

There  are  still  numerous  challenges  that  have  yet  to  be  faced  and  overcome.  First,  Statewide  cooperation  is 
often  hampered  by  politics,  competition  over  limited  resources,  and  mistrust  that  equitable  outcomes  will  result. 
These  assumptions  are  only  just  beginning  to  be  recognized  and  openly  confronted  at  LOFTI  and  other  project 
meetings.  For  example,  in  February  2000  the  broad-based,  Steering  Committee,  looked  at  teamwork  within  its 
various  members.  Without  exception,  a pre-session  survey  revealed  that  individual  members  felt  better  about  their 
own  contributions  toward  participation,  collaboration,  flexibility,  sensitivity,  risk-taking,  commitment,  facilitation, 
and  openness  that  they  did  of  other  members.  They  felt  that  the  Committee  was  only  moderately  effective  as  a high 
performance  team,  and  that  its  members  did  not  know  or  allow  others  to  share  their  individual  talents  and  skills. 
The  descriptor  most  chosen  to  represent  their  feelings  about  participating  was  "challenging,"  and  several  of  the 
survey  comments  were  directed  toward  the  building  of  greater  trust  for  this  learning  organization  to  improve 
(Survey  Results  of  LOFTI  Steering  Committee,  1999). 

Technology  has  often  become  the  focus  and  end  in  and  of  itself,  rather  than  a means  towards  building  our 
future.  Educators  still  struggle  to  translate  the  concepts  and  disciples  for  learning  organizations  from  the  business 
world  to  that  of  education.  The  "bottom  lines,"  "customers,"  and  "productivity  measures"  are  quite  different.  It  is 
still  difficult  to  create  local  visions  of  larger  regional  or  statewide  learning  organizations  when  educators  are  pressed 
to  accomplish  their  daily  tasks  as  efficiently  as  possible  without  tangible  rewards  visible  for  collaboration  and 
sharing.  It  is  also  difficult  to  systematically  build  shared  vision,  personal  mastery,  mental  models,  team  learning, 
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aligned  with  systems  thinking  (Senge,  1990)  and  embracing  features  made  possible  through  new  technology-rich 
learning  environments. 

In  closing,  it  is  an  exhilarating  time  to  be  participating  in  the  creation  of  learning  organizations  within 
education  involving  technology.  This  paper  highlighted  several  initiatives  within  the  State  of  South  Dakota  that  are 
directed  toward  this  end.  It  brought  together  theories  about  leadership  policies  to  create  learning  organizations  that 
have  surfaced  in  the  literature  and  reported  upon  the  progress  and  challenges  faced  by  those  working  with  these 
ideas  in  educational  technology  initiatives  within  the  state  of  South  Dakota  today. 
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Abstract 

This  study  was  designed  to  explore  how  some  students  in  ten  public  schools  view  the  WWW  and  how  their 
attitudes  and  opinions  affect  their  use  of  this  new  medium  in  an  educational  context.  An  exploratory  principal 
components  analysis  of  forty  use  statements  resulted  in  an  eight  factor  solution.  Additionally,  student  responses  to  a 
computer-administered  survey  instrument  were  collected  and  analyzed  revealing  significant  differences  in  the  way 
that  students  describe  their  use  of  the  WWW.  Gender,  grade  level,  and  amount  of  time  spent  using  the  WWW  were 
used  to  create  between-group  comparisons  of  the  WWW  use  categories  that  made  up  the  computer-administered 
survey  instrument.  The  final  phase  of  data  analysis  was  a content  analysis  of  sites  visited  by  students.  A total  of 
123,071  URLs  were  collected  from  the  computers  used  to  administer  the  computer  survey  instrument.  These  were 
reduced  to  a total  of  500  sites  that  were  reviewed  by  media  specialists.  Students  were  found  to  be  visiting 
commercial  sites  at  a much  higher  proportion  than  those  in  other  domains.  Also,  the  commercial  sites  received  the 
lowest  rating  for  " suitability  for  academic  research " of  all  the  domain  names.  And  while  students  reported  their 
purpose  for  using  the  WWW  as  " research  and  learning"  fifty-two  percent  of  the  time,  the  coders  found  only  twenty- 
seven  percent  of  the  sampled  sites  to  be  "suitable”  for  that  purpose. 

Introduction 

This  [PC/Intemet]  technology  promises  to  have  a far  larger  and  more  serious  impact  on  our  society  than  the 
introduction  of  television,  possibly  as  great  an  influence  on  history  as  the  industrial  revolution  or  the  printing  press. 
Television  primarily  involves  only  leisure  time;  this  technology  will  affect  work,  school  and  play — personal,  family 
and  business  relationships.  Surveying  the  Digital  Future:  How  the  PC  and  Internet  Are  Changing  the  World. 

The  introduction  of  a new  medium  into  society  has  frequently  been  a flash-point  for  media  effects  research 
focusing  on  children  and  adolescents  (Wartella  & Reeves,  1985).  In  each  case  children  have  been  recognized  as  a 
special  audience,  one  that  deserves  special  consideration  (Dorr,  1986;  Wartella,  1995).  The  introduction  of 
television  prompted  numerous  studies  (e.g.,  Schramm,  Lyle  & Parker,  1961),  and  provoked  much  discussion  and 
public  debate  over  its  proper  place  in  society.  The  effects  of  the  media  on  children’s  mental  development  has  been  a 
common  theme  (e.g.,  Van  Evra,  1990;  Winn,  1977).  From  early  on,  media  effects  researchers  have  focused  the 
attention  of  the  nation  on  the  media’s  dysfunctional  effects.  From  the  Payne  Fund  studies  on  the  effects  of  motion 
pictures  (e.g.,  Charters,  1933;  Dale,  1935)  to  Dr.  Fredrick  Wertham's  (1954)  expose  of  comic  books  to  the  Surgeon 
General's  (1972)  report  on  television  and  violence,  social  scientiara  have  examined  how  the  media  have  served  to 
undermine  the  positive  influences  of  family  and  social  institutions  (McLeod  & Reeves,  1980;  Wartella  & Reeves, 
1985). 1 

Researchers  exploring  the  effects  of  educational  media,  however,  have  argued  from  a similar  set  of 
assumptions  to  reach  dramatically  different  conclusions.  Instead  of  exploring  the  possible  negative  effects  of  the 
media,  proponents  of  "powerful  effects"  have  heralded  the  positive  effects  promised  by  the  use  of  educational  media 
in  the  classroom  (e.g.,  Kozma,  1994;  Salomon,  1978).  The  history  of  educational  technology,  specifically  the  use  of 
mass  media  in  an  educational  context,  is  infused  with  promises  of  revolutionary  proportions  (Cuban,  1986).  Access 
to  books,  instructional  motion  pictures,  radio,  and  more  recently  television  and  interactive  multimedia  has  been 
envisioned  as  the  panacea  for  all  that  ails  our  educational  system.  The  use  of  the  World  Wide  Web  in  the  classroom 
is  only  the  latest  in  a long  history  of  mass  media  technologies  that  have  been  embraced  by  the  educational 
establishment.  Libraries,  along  with  the  liberating  technologies  of  the  postal  service  and  telephones,  were  once 
envisioned  as  facilitating  the  elimination  of  schools  (Illich,  1970).2  Educational  films,  radio  and  television 
programs,  and  educational  computer  software  have  all  been  employed  with  similar  hope  and  optimism.  The  belief 
that  the  Web  will  lead  to  the  promised  land  is  but  the  most  recent  manifestation  of  this  technological  utopianism. 
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Statement  of  the  Problem 

The  Internet,  and  more  specifically  the  WWW,  is  being  eagerly  adopted  by  school  districts,  administrators, 
and  teachers  almost  without  exception.  However,  the  use  of  the  WWW  in  the  classroom  marks  the  first  deployment 
of  an  educational  medium  in  which  the  end  user  can  access  a virtually  unlimited  breadth  of  content.  Hecht  (1997) 
argued,  "having  the  Internet  in  the  classroom  is  like  equipping  each  classroom  with  a television  that  can  be  turned  on 
at  any  time  and  tuned  in  to  any  of  100,000  unrestricted  channels,  only  a tiny  fraction  of  which  are  dedicated  to 
educational  programming  (and  even  those  have  commercials)."  McNealy  (1999)  voiced  a similar  concern  when  he 
wrote,  "Right  now,  putting  students  in  front  of  Internet  terminals  is  no  better  than  putting  them  in  front  of  TV  sets.  It 
may  even  be  worse"  (p.  17A).  And  while  the  resource  is  huge-Lawrence  and  Giles  (1999)  estimated  800  million  web 
pages-some  have  argued  that  only  a fraction  of  the  millions  of  Web  pages  hold  any  educational  value  for  this  K-12 
audience  (see,  for  example,  Bennett,  Wilkinson,  & Oliver,  1996;  Kirk,  1996;  Tillman,  1998). 

This  raises  some  interesting  questions  regarding  the  use  of  the  WWW  in  an  educational  context.  Even 
students  who  seek  out  educational  content  may  be  thwarted  by  the  very  attributes  heralded  by  WWW  proponents, 
e.g.,  the  WWW's  breadth  and  depth  of  information.  Research  into  selective  exposure,  defined  as  "behavior  that  is 
deliberately  performed  to  attain  and  sustain  perceptual  control  of  particular  stimulus  events,"  (Zillmann  & Bryant, 
1985,  p.  2)  has  raised  questions  about  new  media  technologies  that  provide  an  abundance  of  choice  and  place  even 
greater  control  over  consumption  in  the  hands  of  the  consumer.  In  a nonlinear  medium,  when  educational  content  is 
sometimes  packaged  as  "edu-tainment,"  what  is  to  prevent  students  from  skipping  over  the  "education?"  Preliminary 
data  collected  during  a pilot  study  (Ebersole,  1999)  indicated  that  for  middle  and  high  school  students  accessing  the 
WWW  from  computers  in  public  schools,  the  most  frequently  visited  sites  were  those  that  were  also  the  least 
educational.  And  while  "research"  was  the  most  frequent  response  to  the  question  "why  are  you  using  the  WWW  at 
this  time?"  content  analysis  of  the  sites  visited  suggested  that  "looking  for  something  interesting"  was  the  more 
likely  explanation  for  the  majority  of  sites  visited. 

Significance  of  the  Study 

There  have  been  calls  for  research  to  determine  the  effect  of  computers  with  Internet  access  on  student 
achievement  (Kozma  & Quellmalz,  1996).  Linda  Roberts,  Director  of  the  Office  of  Educational  Technology,  spoke 
at  the  SchoolTech  Exposition  and  Conference  and  reminded  educators  of  the  need  to  collect  data  that  will  support  or 
reject  the  spending  of  billions  of  dollars  for  computers  and  wiring  (Mendels,  1998,  April  27).  A recent  US 
Department  of  Education  report  (National  Center  for  Educational  Statistics,  1997)  concluded  by  listing  four 
challenges  that  remain  for  educators  and  school  districts  as  they  embrace  this  new  educational  technology.  The 
challenges  are:  technical  support  for  hardware  and  software;  teacher  training  and  development;  "increasing  effective 
use  of  the  Internet  to  enhance  student  learning;  and  protecting  students  from  inappropriate  material  on  the  Internet". 
Research  into  the  use  of  the  WWW  in  public  schools  has  important  implication  for  local,  state,  and  national  policy 
and  funding  initiatives.  While  a study  such  as  the  present  one  is  only  a small  step  in  the  quest  to  assess  the  value  of 
this  new  medium,  it  is  important  in  that  it  permits  identification  of  the  motivations  of  individual  users.  The  study  can 
also  help  to  identify  crucial  points  where  intervention  may  be  necessary  in  order  to  realize  the  WWW's  full  potential 
as  an  educational  resource.  Once  we  understand  what  motivates  students  to  utilize  this  medium,  we  can  better  design 
incentives  that  encourage  educational  use  and  discourage  use  that  distracts  students  from  that  goal. 

Theory:  Uses  and  Gratifications 

According  to  Rice  and  Williams  (1984),  "the  new  media  provide  fertile  test  beds  for  many  of  our  theories 
and  models"  (p.  55).  One  mass  media  theory  that  has  repeatedly  been  cited  as  holding  promise  for  the  analysis  of 
new  media  is  uses  and  gratifications.  In  addition  to  the  article  by  Rice  and  Williams,  articles  by  Williams,  Strover 
and  Grant  (1994),  Newhagen  and  Rafaeli  (1996),  Morris  and  Ogan  (1996),  and  December  (1996)  have  also  included 
references  to  the  suitability  of  uses  and  gratifications  for  new  media  research.  As  an  "active  audience"  theory,  uses 
and  gratifications  provides  a vantage  point  from  which  to  look  at  the  ways  that  audiences  respond  to  the  breadth  and 
depth  of  information  that  is  made  available  by  these  new  media.  Newhagen  and  Rafaeli  (1996)  have  suggested  that 
uses  and  gratifications  theory  may  be  especially  useful  because  of  the  "mutability"  of  the  Web,  or  what  Newhagen 
calls  its  "chameleon-like  character"  (p.  11).  The  diversity  of  content  is  much  greater  for  the  WWW  than  for 
traditional  electronic  media.  While  television,  radio,  and  to  a lesser  degree  print  media  are  subject  to  regulatory  and 
societal  scrutiny,  the  WWW  is  virtually  unregulated.  Because  of  this,  the  WWW  literally  has  something  for 
everybody.  The  fact  that  this  range  of  material  is  available  at  school,  library,  workplace,  and  home  would  suggest 
that  potential  uses  for  the  Internet  may  far  exceed  those  provided  by  other  media. 
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Few  studies  have  taken  a uses  and  gratifications  approach  to  studying  the  Internet  and  even  fewer  have 
narrowed  their  focus  to  look  at  the  WWW  (e.g.,  Chamey,  1996;  Stetter,  1997;  Yoo,  1996,  and  Kaye,  1998). 
December  (1996)  identified  "communication,  interaction,  and  information"  as  the  three  broad  categories  for  why 
people  use  the  Internet.  Charney  (1996)  concluded  from  a study  of  university  students  that  the  Internet  is  used  "to 
keep  informed,  for  entertainment  and  diversion,  to  maintain  communication,  and  to  look  at  the  sights  and  sounds  of 
the  'Net"  (p.  88),  but  most  frequently  for  entertainment-diversion  (p.  90).  A 1995  study  of  college  students'  WWW 
usage  resulted  in  "six  motivational  categories:  entertainment,  social  interaction,  passing  the  time,  escape, 
information,  and  Web  site  preference"  (Kaye,  1998,  p.  34).  According  to  the  9th  WWW  User  Survey  conducted  by 
Georgia  Tech  (GVU's  9th  WWW  user  survey,  1998),  the  WWW's  youngest  users  (1 1-20)  use  the  web  mainly  for 
"entertainment"  (81%),  "education"  (70%),  "time  wasting"  (67%),  and  "personal  information"  (60%). 

Methods 

Combining  qualitative  and  quantitative  approaches,  this  study  employed  open-ended  questions,  interviews, 
two  types  of  survey  research,  and  content  analysis  of  WWW  sites  visited  by  students.  Some  of  the  data  was 
collected  with  the  active  participation  of  the  subjects,  while  other  data  was  collected  using  passive  data  collection 
techniques.  Using  multiple  methodologies  allowed  for  increased  richness  of  data  and  a clearer  picture  of  the 
phenomena  under  investigation.3 

Subjects 

The  population  for  this  study  was  comprised  of  middle-school  and  high-school  students  at  selected  public 
schools  in  five  districts  in  a western  state.  The  districts  were  selected  in  consultation  with  the  state's  Department  of 
Education  to  reflect  a cross-section  of  schools  in  urban  and  rural  settings  that  have  Internet  access.  The  participants 
for  this  study  were  selected  using  two  different  approaches.  For  the  first  survey,  which  was  administered  on  paper,  a 
stratified  convenience  sample  was  employed.  At  one  middle  school  and  one  high  school  in  each  district  a class 
representing  each  grade  (sixth,  seventh,  eighth,  ninth,  tenth,  eleventh,  and  twelfth)  was  selected  to  take  the  paper 
survey.  The  second  survey  was  administered  electronically  at  the  computer.  Students  attending  middle  and  high 
schools  in  these  districts  have  access  to  the  WWW  using  computers  available  in  the  schools'  media  centers. 
However,  not  all  students  had  parental  permission  to  access  the  WWW.  Only  students  who  had  been  granted 
parental  permission  and  who  had  signed  and  submitted  the  required  forms  to  their  local  school  administrators  were 
permitted  to  access  the  computer-administered  survey.  Of  these,  participation  in  the  second  phase  of  the  survey  was 
voluntary  and  by  self-selection.  The  survey  was  installed  as  the  default  home  page  in  the  media  centers  for  a period 
of  time  sufficient  to  gather  approximately  1 00  responses  from  each  school. 

Survey  Data  Collection 

The  two  primary  survey  instruments  employed  in  the  current  study  will  be  referred  to  as  the  "paper  survey" 
and  the  "computer  survey."  The  paper  survey  is  a 75-item  survey  instrument  that  was  administered  to  students  in 
their  classrooms  at  selected  public  middle  schools  and  high  schools.  The  paper  survey  contains  sections  designed  to 
measure  the  students' : 1)  affinity  for  the  WWW,  2)  assessment  of  the  value  of  the  WWW  for  various  purposes,  3) 
skill  level  for  computer  and  WWW  use,  4)  use  of  the  WWW,  5)  avoidance  of  the  WWW,  and  6)  demographics. 
Following  this,  the  computer  survey  was  administered  to  the  students  at  the  time  and  place  of  their  access  to  the 
WWW-specifically  the  school's  media  center  or  library.  The  computer  survey  is  comprised  of  just  four  questions: 
grade,  gender,  how  much  the  student  uses  the  WWW,  and  the  student's  purpose  for  using  the  WWW  at  this 
particular  time.  This  survey  was  intentionally  kept  very  short  in  order  to  prevent  student  frustration  and  a perception 
of  "time-off-task"  that  may  have  jeopardized  the  support  of  school  administrators. 

Passive  Data  Collection  Design  for  This  Study 

December  (1996)  and  Newhagen  and  Rafaeli  (1996)  recognized  the  fact  that  the  Internet  provides  excellent 
opportunities  for  data  collection.  As  Rafaeli  noted,  any  social  scientist  who  has  looked  at  an  Internet  server  must  be 
struck  by  the  research  possibilities  present  in  the  data  that  is  passing  through  that  computer  (p.  6).  In  order  to  take 
advantage  of  this  unique  feature  of  the  WWW,  the  design  of  this  study  calls  for  passive  data  collection  to  follow  the 
survey  research.  Both  Netscape  Navigator  and  Microsoft's  Internet  Explorer  browsing  software  generate  a cache  or 
"global  history"  file  that  resides  on  the  user's  hard  drive  and  which  retains  a list  of  addresses  (Uniform  Resource 
Locators  or  URLs)  of  WWW  sites  last  visited.  This  list  of  URLs  listing  WWW  pages  and  graphics  visited  most 
recently  is  extensive  and  can  be  thousands  of  sites  long. 
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At  the  beginning  of  the  data  collection  phase  the  cache  files  on  the  computers  in  the  schools'  Media  Centers 
were  deleted.  At  the  end  of  the  collection  period  the  cache  files  were  copied  to  a disk  and  the  data  prepared  for 
analysis.  A total  of  123,071  URLs  were  collected  from  the  more  than  80  Macintosh  and  Windows  personal 
computers  on  which  the  survey  instrument  had  been  installed.  First,  the  number  of  occurrences  of  web  sites  from  the 
five  generic  top-level  domains  (commercial  [.com],  educational  [.edu],  governmental  [.gov],  network  [.net],  and 
organizational  [.org]),  and  the  United  States  (.us)  domain  was  recorded.  In  order  to  facilitate  content  analysis  of  the 
sites  visited,  URLs  ending  with  .gif  and  .jpg  were  first  stripped  from  the  list  and  then  a UNIX  grep  script  was 
written  and  applied  to  the  remaining  sites  to  reduce  the  list  to  the  number  of  randomly  selected  sites  that  could  be 
evaluated  and  coded  given  the  time  and  resources  available.  The  subsequent  500  URLs  were  then  collected  into  a 
single  WWW  page  and  two  educators/media  specialists,  one  male  and  the  other  female,  from  a nearby  school  district 
were  asked  to  analyze  these  WWW  pages  and  serve  as  evaluators.  The  pages  were  assigned  a "use"  category  based 
on  the  same  choices  that  had  been  presented  to  the  students  on  the  computer-administered  survey  and  were  rated  for 
"suitability  as  a source  for  academic  research"  on  a scale  of  1-3:  1 = not  suitable,  2 = questionable,  and  3 = suitable. 
The  evaluators  were  instructed  to  look  at  each  WWW  page  with  consideration  for  the  grade  level  of  the  students 
being  studied. 

Results 

Respondents  to  the  paper  survey  (n  = 791)  ranged  in  age  from  10-21  years  (M  = 14.45)  and  were  enrolled 
in  the  6th  grade  (12%),  7th  grade  (130,  17%),  8th  grade  (123,  16%),  9th  grade  (14%),  10th  grade  (15%),  1 1th  grade 
(12%),  and  the  12th  grade  (16%).  Average  self-reported  grade  point  average  (GPA)  was  3.28,  and  51%  were  male. 
Ethnicity  of  respondents  is  as  follows:  American  Indian  (2%),  Asian  (3%),  Black  (9%),  Hispanic  (16%),  White 
(69%),  and  other  (2%).  Respondents  to  the  computer-administered  survey  (n  = 1083)  were  enrolled  in  the  6th  grade 
(5%),  7th  grade  (21%),  8th  grade  (19%),  9th  grade  (14%),  10th  grade  (15%),  1 1th  grade  (12%),  and  the  12th  grade 
(16%).  Of  these,  59%  were  male. 

Attitudes  towards  the  WWW 

Questionnaire  items  1 through  5 were  statements  designed  with  the  goal  of  determining  the  students' 
affinity  for  the  WWW.  This  was  operationalized  by  summing  five  Likert-scale  responses  with  results  ranging  from 
5-25  (M  = 12.57,  SD  = 4.12,  Cronbach's  alpha  = .83).  The  statement  "Using  the  WWW  is  very  important  to  me" 
received  the  strongest  support  (M  = 3.05)  while  the  statement  "I  would  feel  lost  without  the  WWW"  received  the 
least  (M  = 1.97). 

The  next  set  of  items  was  designed  to  explore  the  students'  beliefs  about  the  WWW-in  particular  the 
WWW's  value  as  a source  of  information,  entertainment,  and  as  a means  of  communication.  As  a source  of 
information  (M  = 1.68,  1 = excellent,  4 = poor),  students  rated  the  WWW  as  "excellent"  44%  of  the  time,  "good" 
46%  of  the  time,  "fair"  9%  of  the  time,  and  "poor"  2%  of  the  time.  As  a source  of  entertainment  (M  = 1 .90),  students 
rated  the  WWW  as  "excellent"  36%  of  the  time,  "good"  42%  of  the  time,  "fair"  1 8%  of  the  time,  and  "poor"  4%  of 
the  time.  And  as  a means  of  communication  (M  = 1.78),  students  rated  the  WWW  as  "excellent"  42%  of  the  time, 
"good"  42%  of  the  time,  "fair"  14%  of  the  time,  and  "poor"  3%  of  the  time.  Based  on  these  responses,  students  rate 
the  WWW  highest  for  information,  followed  by  communication,  and  then  entertainment. 

The  most  common  response  to  a general  question  asking  overall  skill  at  using  computers  was  "good" 
(47%),  followed  by  "average"  (30%),  "excellent"  (20%),  and  "below  average"  (3%).  Additional  questions  asked 
students  how  long  they  have  being  using  the  WWW,  how  many  times  per  week,  and  how  many  hours  per  week  they 
use  the  WWW.  To  the  question,  "For  how  long  have  you  been  using  the  World-Wide  Web?"  the  most  common 
response  was  "1-2  years"  (32%),  followed  by  "more  than  2 years"  (29%),  "6  mo.-l  year"  (23%)  and  "less  than  6 
months"  (17%).  To  the  question,  "Approximately  how  many  times  per  week  do  you  use  the  World-Wide  Web?"  the 
most  common  response  was  "1-2  times"  (35%),  followed  by  "less  than  1"  (28%),  "3-5  times"  (23%),  and  "more  than 
5 times"  (15%).  And  in  response  to  the  question  "About  how  many  hours  per  week  do  you  use  the  World-Wide 
Web?"  respondents  answered  "1-2  hours"  (34%),  "less  than  1"  (33%),  "3-5  hours"  (20%),  "6-10  hours"  (9%),  and 
"10+  hours"  (5%). 

An  additional  four  items  assessed  the  respondent's  skill  at  using  the  WWW.  Students'  self-reported  skill  at 
using  the  WWW  was  operationalized  as  the  sum  of  responses  to  four  Likert-scales.  Skill  at  using  the  WWW  ranged 
from  4 to  20  (M  = 13.86,  SD  = 3.25,  Cronbach's  alpha  = .78). 

Students  were  also  asked  to  indicate  the  locations  where  they  access  the  WWW.  The  625  students  who 
responded  to  question  20  indicated  the  following  places  were  used  to  access  the  WWW;  home  (69%),  school  (61%), 
friend's  house  (32%),  public  library  (23%),  and,  other  (1 1%). 
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Reasons  for  using  the  WWW 

Questionnaire  items  21  through  60  addressed  reasons  why  students  might  choose  to  use  the  WWW.  These 
items  were  generated  from  statements  made  by  middle  school  and  high  school  students  who  responded  anonymously 
to  an  open-ended  question  asking  them  to  list  several  things  "that  the  World-Wide  Web  is  good  for."  Additional  use 
statements  were  taken  from  fill-in-the-blank  responses  to  the  computer  survey  questionnaire  in  the  pilot  study. 

Because  of  the  paucity  of  research  in  uses  and  gratifications  of  the  relatively  new  WWW,  exploratory 
factor  analysis  (SPSS  Principal  Components  Analysis  with  Varimax  rotation)  was  employed  to  group  these  use 
statements  into  categories.  A preliminary  principal  components  analysis  was  performed  on  an  incomplete  data  set  in 
order  to  arrive  at  a list  of  "use  statements"  that  became  part  of  the  computer-administered  survey  instrument.  Those 
seven  use  statements  were:  "for  research  and  learning,"  "to  communicate  with  other  people,"  "for  access  to  material 
otherwise  unavailable,"  "to  find  something  fun  or  exciting,"  "for  something  to  do  when  I'm  bored,”  "for  sports  and 
game  information,"  and,  "for  shopping  and  consumer  information."  Once  the  complete  data  set  was  collected  via  the 
paper  survey  instrument,  another  principal  components  analysis  was  conducted.  This  time  the  result  was  eight 
factors  with  eigenvalues  greater  than  1.0  accounting  for  a total  of  58%  of  the  total  variance.  These  factors  differed 
slightly  from  those  derived  from  the  earlier  analysis.4  (see  Table  1). 

Table  1 


Items 

Factor  1:  Research  and  Learning 

(eigenvalue  = 10.57,  variance  after  rotation  = 14%) 

Because  it  is  a good  source  of  news 

To  complete  homework  assignments 

To  get  up-to-date  facts  and  information 

To  leam  how  to  use  computers  better 

To  leam  new  things  t 

Because  it  provides  a new  outlook  on  learning 

Because  it’s  educational 

To  find  articles  and  references 

Because  it  is  an  excellent  source  of  information 

To  find  out  what’s  going  on  in  the  world 

So  that  I can  do  better  in  school 

To  conduct  research  for  class 

Factor  2:  Easy  Access  to  Entertainment 

(eigenvalue  = 3.73,  variance  after  rotation  = 9%) 

Because  it’s  so  easy 

Because  it’s  fun 

To  leam  new  things  t 

To  play  games  t 

Because  it’s  exciting 

For  entertainment  t 

To  find  interesting  things 

Because  computers  are  cool  t 

Factor  3:  Communication  and  Social  Interaction 
(eigenvalue  = 2.19,  variance  after  rotation  = 9%) 

To  chat  with  other  people 
To  find  people 

So  that  I can  have  foreign  language  friends 
To  email  friends 

To  talk  with  people  from  around  the  world 
To  meet  new  people 


FAC1 

FAC2 

FAC3 

FAC4 

FAC5 

FAC6 

FAC7 

FAC8 

.488 

.237 

-.048 

.117 

.266 

.141 

-.165 

.238 

.553 

.063 

.114 

.031 

.296 

-.336 

.392 

.047 

.445 

.195 

.026 

.080 

.214 

.224 

-.117 

.397 

.423 

.307 

.172 

-.095 

-.019 

.206 

.195 

.074 

.501 

.554 

.140 

.019 

.132 

.045 

.009 

.082 

.727 

.264 

.149 

.000 

.067 

.113 

-.045 

.015 

.749 

.237 

.054 

-.071 

-.024 

.110 

-.047 

-.052 

.675 

-.115 

-.008 

.035 

.270 

.241 

-.086 

.071 

.682 

.118 

.085 

.126 

.233 

.025 

-.090 

.056 

.566 

.175 

.227 

.269 

-.058 

.256 

-.037 

.196 

.763 

.190 

.140 

.077 

-.048 

-.014 

.077 

.007 

.766 

-.061 

.009 

.026 

.214 

-.046 

.013 

-.046 

.185 

.521 

.049 

.203 

.286 

-.007 

-.161 

.256 

.094 

.746 

.129 

.281 

.148 

.037 

.091 

.091 

.501 

.554 

.140 

.019 

.132 

.045 

.009 

.082 

.036 

.434 

.102 

.325 

.028 

.030 

.537 

.068 

.238 

.712 

.180 

.177 

.063 

.084 

.130 

-.001 

-.002 

.508 

.212 

.467 

.149 

.076 

.190 

.077 

.351 

.573 

.117 

.121 

.206 

.237 

-.007 

.024 

.226 

.453 

.105 

.428 

-.075 

.331 

.094 

-.096 

-.030 

.033 

.716 

.220 

.241 

-.103 

.161 

.122 

.086 

.101 

.637 

.056 

.104 

.027 

.158 

.232 

.168 

.232 

.594 

-.034 

-.129 

.221 

-.089 

.110 

.096 

-.005 

.520 

.263 

.279 

.082 

-.079 

.184 

.145 

.156 

.788 

.061 

-.020 

.123 

.038 

.011 

.116 

.106 

.801 

.165 

-.003 

.135 

.009 

.063 

106 
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Factor  4:  Something  to  Do  When  Bored 
(eigenvalue  = 1.68,  variance  after  rotation  = 7%) 

For  entertainment  t 
For  browsing 

Because  it  gives  me  something  to  do 

When  I’m  bored 

Because  computers  are  cool  t 

Factor  5:  Access  to  Material  Otherwise  Unavailable 
(eigenvalue  = 1.53,  variance  after  rotation  = 6%) 

To  find  things  not  in  the  library 

Because  it  is  convenient 

To  download  software  and  other  free  stuff  t 

Because  I can  access  things  otherwise  unavailable  to  me 

Factor  6:  Product  Info  and  Tech  Support 
(eigenvalue  =1.17,  variance  after  rotation  = 5%) 

To  download  software  and  other  free  stuff  f 
To  get  product  information 
To  get  information  about  games  t 
To  get  technical  support 

Factor  7:  Games  and  Sexually  Explicit  Sites 
(eigenvalue  = 1.09,  variance  after  rotation  = 5%) 

To  get  sports  information  and  statistics  f 
To  access  sexually  explicit  sites 
To  play  games  t 

To  get  information  about  games  f 

Factor  8:  Consumer  Transactions 
(eigenvalue  = 1 .08,  variance  after  rotation  = 4%) 

For  shopping  and  making  purchases 
To  look  up  music  and  concert  information 

To  get  sounds,  pictures,  or  animations  for  projects  * 


-.002 

.508 

.212 

.467 

.149 

.076 

.190 

.077 

.266 

.185 

.109 

.510 

.254 

.239 

-.007 

-.022 

.018 

.258 

.186 

.761 

.055 

.045 

.100 

.104 

-.003 

.152 

.155 

.796 

.085 

-.017 

.064 

.106 

.226 

.453 

.105 

.428 

-.075 

.331 

.094 

-.096 

.240 

.060 

-.020 

-.004 

.652 

-.041 

-.063 

oo 

co 

CD 

.143 

.198 

.131 

.264 

.537 

.041 

-.088 

.170 

-.115 

.177 

.217 

.042 

.425 

.438 

.302 

.153 

.167 

.178 

.119 

.086 

.617 

.280 

.067 

.020 

-.115 

.177 

.217 

.042 

.425 

.438 

.302 

.153 

.197 

.102 

.136 

.040 

.245 

.489 

.044 

.125 

.011 

.292 

.067 

.272 

-.012 

.467 

.511 

-.002 

.348 

.097 

.261 

.127 

.064 

.610 

.148 

.096 

.243 

-.002 

-.018 

.072 

-.072 

.194 

.439 

.413 

-.191 

-.001 

.058 

-.001 

-.032 

.055 

.672 

.077 

.036 

.434 

.102 

.325 

.028 

.030 

.537 

.068 

.011 

.292 

.067 

.272 

-.012 

.467 

.511 

-.002 

.001 

.023 

.247 

.054 

-.015 

.089 

.107 

.701 

.020 

.138 

.268 

.084 

.102 

-.010 

.092 

.674 

.259 

.360 

.077 

.091 

.366 

.097 

.180 

.199 

.887 

.844 

.823 

.813 

.601 

.653 

.617 

.577 

Cronbach  Alpha  (for  scale) 


* Denotes  item  that  did  not  meet  the  criteria  for  factor  loading 
t Denotes  item  that  loaded  highly  on  two  factors 


Correlation  Analyses 

Following  the  independent-sample  t tests,  Pearson  product-moment  correlation  coefficients  were  computed 
among  the  eight  WWW  use  scales  and  three  variables  measured  at  the  interval  level.  The  three  variables  that  were 
correlated  with  the  eight  factors  were  grade,  affinity  for  the  WWW,  and  skill  level  at  using  the  WWW.  Not 
surprisingly,  both  affinity  and  skill  are  positively  correlated  with  every  type  of  use  as  defined  by  this  study.  The  few 
significant  correlations  between  grade  and  uses  suggest  that  students  in  the  lower  grades  are  more  likely  to  say  that 
they  use  the  WWW  for  easy  access  to  entertainment,  for  something  to  do  when  bored,  for  product  information  and 
technical  support,  and  for  games  and  sexually  explicit  sites.  Likewise,  students  in  the  higher  grades  are  more  likely 
to  say  that  they  use  the  WWW  for  access  to  material  otherwise  unavailable.  A significant  negative  correlation  was 
obtained  between  grade  level  and  affinity  for  the  WWW  (r  = -.15,  p = .001).  This  suggests  that  as  respondents 
increase  in  age  and  grade  level,  they  become  less  enamored  of  the  WWW.  Note,  however,  that  this  reduction  in 
affinity  for  the  WWW  does  not  appear  to  result  in  less  use  with  increased  age.  The  correlation  between  grade  level 
and  skill  level  was  not  significant  (r=  -.02,  p = .717).  The  lack  of  correlation  between  grade  and  skill  level  may  be 
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accounted  for  by  the  likelihood  that  students  rated  their  skill  level  against  that  of  their  peers  rather  than  against  an 
objective  scale. 

Reasons  for  avoiding  the  WWW 

In  addition  to  seeking  reasons  why  students  choose  to  use  the  WWW  in  a school  setting,  questions  were 
asked  in  an  attempt  to  discover  possible  reasons  why  students  would  choose  to  avoid  using  the  WWW.  Ten 
avoidance  statements  were  presented  to  students  who  were  asked  to  respond  on  the  same  agree/disagree  scale.  An 
analysis  of  the  avoidance  statements  suggested  that  face-to-face  interaction  with  peers  is  the  leading  reason  given  for 
not  spending  time  using  the  WWW.  Other  important  factors  included  several  statements  about  negative  issues 
commonly  attributed  to  the  WWW,  e.g.,  pornography,  illegal  activity,  and  other  users  who  may  have  suspect 
motives.  And  while  schools  often  have  fast  connections  to  the  Internet,  these  users  appeared  to  be  consistent  with 
the  general  population  of  aWWW  users  who  perceive  the  WWW  to  be  too  slow,  especially  when  downloading 
graphically  intensive  sites.  On  a positive  note,  the  social  stigma  that  has  been  associated  with  computer  expertise- 
i.e.,  that  "computers  are  for  nerds"-does  not  appear  to  carry  much  weight  with  these  respondents. 

Results  of  the  Computer-Administered  Survey 

Unlike  the  nearly  even  split  by  gender  in  the  paper  survey,  the  respondents  to  the  computer-administered 
survey  were  slightly  more  than  59%  male.  This  could  indicate  that  males  are  heavier  users  of  the  WWW  at  school, 
or  that  they  were  more  likely  to  respond  to  the  survey.  While  the  breakdown  of  respondents  by  grade  level  to  the 
paper  and  computer  surveys  was  identical  for  high  school  students,  there  was  a substantial  difference  in  the  number 
of  sixth,  seventh,  and  eighth  grade  students  responding  to  the  two  surveys.  The  reduced  percentage  of  sixth  grade 
students  responding  to  the  computer-administered  survey  was  explained  in  part  by  the  fact  that  one  district  did  not 
provide  Internet  access  to  sixth  grade  students.  Because  there  was  no  way  to  prohibit  multiple  responses  from 
students  responding  to  the  computer-administered  survey,  the  data  reported  here  should  not  be  interpreted  as 
representing  unique  students,  but  rather  as  sessions  at  a computer. 

In  response  to  the  question  asking  the  average  amount  of  time  spent  using  the  WWW  each  week,  the  most 
common  response  was,  "less  than  1 hour  per  week"  (36%)  followed  by  "1  to  2 hours  per  week"  (23%),  "3  to  5 hours 
per  week"  (18%),  "more  than  10  hours  per  week"  (17%),  and  "6  to  10  hours  per  week"  (6%).  Respondents  to  the 
computer-administered  survey  gave  the  following  reasons  for  using  the  WWW:  "for  research  and  learning"  (n  = 541, 
52%),  "to  communicate  with  other  people"  (n  = 74,  7%),  "for  access  to  material  otherwise  unavailable"  (n  = 55, 
5%),  "to  find  something  fun  or  exciting"  (n  = 85,  8%),  "for  something  to  do  when  I'm  bored"  (n  = 56,  5%),  "for 
sports  and  game  information"  (n  = 65,  6%),  and,  "for  shopping  and  consumer  information"  (n  = 10,  1%).  In  addition, 
165  students  (16%)  chose  not  to  select  from  the  seven  options  presented.  Of  these,  94  students  elected  to  write-in  a 
response  to  this  question.  The  write-in  responses  offered  by  students  to  explain  their  purpose  for  using  the  WWW 
were  grouped  into  categories  as  follows:  specific  research  topics  (n  =20),  sexually  explicit  material  (n  =20),  games 
and  amusements  (n  =14),  general  research  and  learning  (n  =11),  combinations  of  things  (n  =10),  communication  (n 
=5),  and  other  unclassified  (n  =14). 

Content  Analysis  of  Sites  Visited 

Because  there  are  normative  expectations  for  media  content  consumed  in  school  for  educational  purposes, 
content  analysis  of  sites  visited  by  students  was  employed  to  better  understand  the  nature  of  the  content  being 
consumed.  Of  the  123,071  URLs  collected,  77%  (n  = 94,426)  were  from  the  .com  domain,  5%  (n  = 6,289)  were 
from  .net,  5%  (n  = 5,704)  were  from  .org,  4%  (n  = 4,842)  were  from  .edu,  1 percent  (n  = 1,640)  were  from  .gov,  1% 
(n  = 1,403)  were  from  .us,  and  7%  (n  = 8,767)  were  from  another  or  unidentified  domain  name.  These  numbers 
stand  in  contrast  to  the  distribution  of  domain  names  that  makes  up  the  current  state  of  the  WWW.  According  to  the 
latest  survey  of  WWW  domain  names  by  host  count,  conducted  by  Internet  Software  Consortium  in  January  of 
2000,  the  actual  make-up  of  the  WWW  is  not  as  heavily  skewed  towards  the  commercial  domain  sites  as  the  student 
sample  would  suggest  (see  Table  2). 
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Table  2 

Domain  Names  by  Host  Count 


Domain  Name 

Number  of  sites* 

% of  total 

% of  123,071  Web  pages 

(com,  net,  edu,  us,  org,  gov) 

visited  by  students 

com  (commercial) 

29075185 

50.4 

77 

net  (network) 

18305485 

31.7 

5 

edu  (education) 

6313781 

10.9 

4 

J'P 

2680659 

uk 

2240216 

us 

2062653 

3.6 

1 

mil 

1908413 

de 

1778831 

ca 

2153807 

au 

1181376 

org  (organization) 

1063901 

1.8 

5 

nl 

839912 

fr 

867981 

gov  (government) 

842854 

1.5 

1 

Total 

57663859 

99.9% 

93%  (7%  = other) 

* Source:  Internet  Software  Consortium,  January  2000 


The  next  step  was  a more  detailed  content  analysis  of  randomly  selected  WWW  pages  visited  by  students. 
Once  intercoder  reliability  was  established  at  an  adequate  level  (alpha  = .92),  the  500  randomly  selected  WWW 
pages  were  analyzed  for  educational  value  and  use  category.  The  "suitability"  rank  most  often  assigned  by  the 
evaluators  was  "not  suitable"  (n  = 262,  57%),  followed  by  "suitable"  (n=  135,  29%),  and  "questionable"  (n  = 65, 
14%).  The  use  category  most  often  assigned  by  the  evaluators  was  "for  research  and  learning"  (n  = 126,  27%), 
followed  by  "access  to  material  otherwise  unavailable"  (n  = 102,  22%),  "to  find  something  fun  and  exciting"  (n  = 
76,  17%),  followed  by  "for  shopping  and  consumer  information"  (n  = 61,  13%),  "for  something  to  do  when  I'm 
bored"  (n  = 45,  10%),  "for  sports  and  game  information"  (n  = 30,  7%),  and  finally,  "to  communicate  with  other 
people"  (n  = 22,  5%). 

When  compared  to  the  use  categories  self-reported  by  the  students  responding  to  the  computer- 
administered  survey  there  is  clearly  a disparity  between  the  way  that  students  and  media  specialists  view  the  content 
and  potential  use  of  these  WWW  sites.  Students'  self-reported  uses  of  the  WWW  was  as  follows:  "for  research  and 
learning"  (n  = 541,  52%),  followed  by  "to  find  something  fun  and  exciting"  (n  = 85,  8%),  "to  communicate  with 
other  people"  (n  = 74,  7%),  "for  sports  and  game  information"  (n  = 65,  6%),  "for  something  to  do  when  I'm  bored" 
(n  = 56,  5%),  "for  access  to  material  otherwise  unavailable"  (n  = 55,  5%),  and  finally,  "for  shopping  and  consumer 
information"  (n  = 10,  1%). 

The  disparity  between  self-reported  uses  of  the  WWW  and  evaluators'  assessments  of  actual  sites  visited 
invites  several  possible  explanations.  First,  as  an  audience-centered  theory  of  media  use,  uses  and  gratifications 
allows  for  individual  interpretation  of  content.  It  should  not  be  surprising  that  students  and  media  specialists 
frequently  envision  different  uses  for  the  same  Web  site.  Second,  students  may  be  responding  to  the  survey  with 
answers  that  they  believe  are  socially  acceptable.  Even  with  the  anonymity  provided  by  the  computer  survey 
technique,  students  may  feel  some  pressure  to  respond  in  a manner  that  is  congruent  with  the  stated  purpose  of  the 
WWW  in  school  as  elaborated  in  the  school  district's  Acceptable  Use  Policy-namely,  academic  research.  However, 
there  may  be  another  factor  at  work  here.  It  could  be  that  students  are  starting  out  with  the  intention  to  conduct 
academic  research,  but  are  finding  themselves  frustrated  or  distracted  by  the  other  offerings  so  readily  available  on 
the  WWW. 

The  analysis  comparing  domain  and  "suitability  for  academic  research"  indicated  low  ratings  for  .com  and 
.net,  with  higher  values  for  .org  and  .gov  (see  Table  3).  In  fact,  the  most  frequently  visited  domain  name  (.com)  had 
the  lowest  educational  value  and  one  of  the  least  frequently  visited  domain  names,  (.gov),  had  the  highest 
educational  value  as  determined  by  the  evaluation  of  the  media  specialists. 
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Table  3 

Mean  Suitability  for  Academic  Research  of  Sites  by  Leading  Domain  Names 


Domain  Name 

(N) 

Mean  suitability  for  academic 
research  as  assigned 
by  coders  * 

.com 

(410) 

1.59 

•org 

(25) 

2.78 

.edu 

(16) 

2.44 

.net 

(12) 

1.75 

■gov 

(9) 

3.0 

.us 

(5) 

2.0 

other 

(23) 

1.94 

Note:  * 1 = not  suitable,  2 = questionable,  3 = suitable 


Discussion 

In  order  for  educators  and  researchers  to  evaluate  the  benefit  of  WWW  use  by  students  in  public  schools  it 
is  necessary  to  begin  by  attempting  to  understand  how  students  perceive  the  WWW,  what  they  use  it  for,  and  what 
gratifications  they  receive  during  periods  of  access.  This  research  explored  the  active  and  goal-directed  use  of  the 
WWW  by  middle  school  and  high  school  students  in  10  public  schools  located  in  five  public  school  districts. 
Consistent  with  the  study’s  exploratory  nature  the  results  included  the  identification  of  gratifications  sought  from 
this  new  electronic  interactive  medium  within  a school  setting.  While  this  study  did  not  provide  answers  to  questions 
about  the  effect  of  WWW  usage  on  student  performance  it  did  provide  answers  to  a more  fundamental  set  of 
questions.  Two  survey  instruments  were  used  to  assess  the  following:  students’  affinity  for  the  WWW,  the  amount  of 
time  spent  using  the  WWW,  students’  self-assessed  skill  level,  their  beliefs  about  the  relative  value  of  the  WWW  as 
a source  of  both  information  and  entertainment  and  as  a means  of  communication,  and  their  reasons  for  using  or  not 
using  the  WWW.  And  finally,  student  use  of  the  WWW  was  monitored  by  sampling  actual  sites  visited  and  by 
content-analyzing  these  sites  for  educational  value. 

Granted  there  is  much  excellent  information  available  on  the  WWW.  But  as  this  study’s  content  analysis  of 
sites  visited  seems  to  indicate,  students  frequently  are  either  not  looking  for  it,  or  if  they  are,  are  unsuccessful  in 
finding  it.  It  is  quite  possible  that  users  who  approach  a Web  search  with  instrumental  intentions  may  soon  find 
themselves  distracted  by  the  entertaining  and  diverting  offerings  available.  The  ease  with  which  one  can  travel  to 
any  comer  of  the  vast  Web  can  be  both  a blessing  and  a curse.  And  this  is  not  a unique  observation.  A study  of  6th 
and  9th  grade  science  students  found  that  without  substantial  guidance  and  assistance,  students  were  often 
unsuccessful  in  locating  useful  academic  information  on  the  WWW  (Lyons,  Hoffman,  Krajcik,  & Soloway,  1997). 
According  to  the  researchers,  ’’one  overall  theme  is  clear  from  the  data:  students  need  a tremendous  amount  of 
support  to  be  successful  in  online  inquiry’’  (p.  12). 
o 

Commercialization  of  the  WWW 

The  commercialization  of  the  WWW  and  the  growth  of  online  advertising  is  another  area  of  growing 
concern  for  educational  technology  advocates.  The  disproportionate  use  of  commercial  WWW  sites  by  students  in 
this  study  is  just  one  indicator  of  the  potential  exposure  of  children  to  advertising.  Students  are  often  unaware  of  the 
questionable  nature  and  value  of  information  from  commercial  WWW  sites.  Lyons  et  al.  (1997)  found  that  students 
’’often  choose  a commercial  site  (.com  in  the  URL)  over  a government  (.gov)  or  education  (.edu)  site”  (p.  21-22),  an 
observation  supported  by  this  research. 

While  unregulated  advertising  itself  is  cause  for  concern,  the  unique  capability  afforded  by  the  WWW  to 
collect  user  information  and  track  usage  raises  even  greater  concern.  The  Zap  Me  corporation  announced  a program 
to  give  free  computers  with  satellite-based  Internet  service  to  schools  in  exchange  for  the  opportunity  to  include 
advertising  in  a comer  of  the  monitor.  In  a New  York  Times  article  dated  February  25,  1999,  Richtel  described  the 
Zap  Me  system,  which  tracks  the  user’s  "grade  level,  sex  and  Zip  code”  in  order  to  "dish  out  age  and  sex-appropriate 
advertisements"  (p.  G7).  At  the  time  of  the  article  the  company  had  "given  computers  to  55  schools  in  eight  states." 
This  kind  of  targeted  advertising  and  the  collection  of  user  data  for  commercial  purposes  should  be  of  concern  not 
only  to  privacy  advocates  but  also  to  educators  and  parents  who  are  concerned  about  the  negative  effects  of 
consumer-driven  culture. 


o 
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Socialization 

More  than  a decade  ago  Rosengren  and  Windahl  (1989)  wrote, 
today's  moral  panics  about  videos,  cable,  satellites,  computer  games  and  the  like,  may  concern  rather  ephemeral 
phenomena.  It  may  be  true  that  for  some  time  media  novelties  may  have  a capacity  to  spell-bind  children,  preventing 
them  from  other,  perhaps  better,  activities.  But  it  is  probably  also  true  that  such  an  influence  will  be  transient  and 
will  be  greatly  reduced  or  even  vanish  as  the  "new"  media  find  their  place  in  society,  (p.  250) 

While  some  of  the  moral  panic  spoken  of  by  Rosengren  and  Windahl  may  have  faded,  it  appears  obvious  that  the 
"new"  medium  of  the  moment,  the  WWW,  has  simply  attracted,  and  perhaps  amplified,  the  concern  that  was  once 
focused  on  the  old  media.  According  to  Stephen  Kerr,  professor  in  the  College  of  Education  at  the  University  of 
Washington, 

The  Net's  beauty  is  that  it’s  uncontrolled...  It's  information  by  anyone,  for  anyone.  There’s  racist  stuff, 
bigoted,  hate-group  stuff,  filled  with  paranoia;  bomb  recipes;  how  to  engage  in  various  kinds  of  crimes,  electronic 
and  otherwise;  scams  and  swindles.  It's  all  there.  It's  all  available...  That's  the  antithesis  of  what  classroom  kids 
should  be  exposed  to.  (quoted  in  Oppenheimer,  1997,  p.  61) 

Concern  about  inappropriate  material  available  on  the  WWW  is  one  reason  for  the  widespread  use  of 
acceptable  use  policies.  Perhaps  the  issues  of  greatest  concern  focus  on  the  presence  of  hate  speech,  sites  promoting 
violent  behavior  and  the  means  to  carry  out  violent  activities,  pornography,  and  sexually  explicit  material.  The 
unregulated  nature  of  the  WWW  and  its  diversity  of  content  providers  serves  to  ensure  the  widest  possible  range  of 
content.  While  other  educational  media  are  controlled  by  the  school  teachers  and  administrators  who  make  decisions 
about  what  books,  videos,  and  magazines  to  place  in  the  media  collection,  the  unfiltered  WWW  has  been  made 
available  to  students  who  frequently  operate  on  an  honor  code  of  self-regulation. 

Limitations 

The  most  obvious  limitation  of  this  study  is  the  sample  and  the  methodology  employed  in  its  selection.  Use 
of  non-probabilistic  sampling  for  schools  chosen  and  self-selection  of  students  taking  the  computer-administered 
surveys  are  acknowledged  as  restricting  the  generalizability  of  this  study.  Because  of  the  non-random  nature  of  the 
sample,  generalization  to  the  larger  population  of  US  adolescents  is  discouraged.  However,  this  study  aided  in  the 
identification  of  several  motivations  for  use  of  the  WWW  at  school  and  these  offer  heuristic  value  for  future 
research. 

As  Chamey  (1996)  found  using  a similar  approach  to  study  college  students'  use  of  the  WWW,  a 
complicating  factor  is  that  student  use  of  this  medium  is  confounded  by  use  that  has  been  assigned  by  a teacher. 
Unlike  uses  and  gratifications  studies  of  other  media,  use  of  the  WWW,  especially  in  a school  setting,  is  a mixture  of 
uses  motivated  by  personal  interest  as  well  as  those  prescribed  by  authority  figures.  In  this  study  there  was  no  way  to 
differentiate  student  use  that  was  self-motivated  versus  use  that  was  encouraged  or  even  mandated  by  teachers. 

Also,  because  of  the  need  to  protect  the  anonymity  of  respondents,  students  were  not  identified  in  a way 
that  allowed  comparisons  between  the  two  survey  instruments.  Neither  were  comparisons  possible  between  the 
responses  to  the  survey  instruments  and  the  content  analysis  of  WWW  sites  visited  by  students.  A research  design 
that  allowed  for  anonymous  tracking  of  responses  from  one  survey  instrument  to  the  next,  and  then  tied  WWW  sites 
visited  to  a specific  anonymous  respondent,  would  have  added  heuristic  value  to  the  study. 

On  a related  note,  the  analysis  of  sites  visited  by  students  did  not  take  into  account  sites  that  were  visited 
accidentally  or  for  only  a short  period  of  time.  It  is  quite  possible  that  sites  ranking  low  on  "suitability  for  academic 
research"  were  visited  only  briefly  while  more  suitable  sites  were  visited  for  longer  periods  of  time,  or  even  printed 
for  later  use. 

Conclusions 

In  the  opening  chapter  of  Failure  to  Connect,  Healy  (1998)  stated:  "Today’s  children  are  the  subjects  of  a 
vast  and  optimistic  experiment"  (p.  17).  Referring  to  the  use  of  computers  for  educational  purposes,  Healy  argued 
that  computers  raise  more  questions  than  they  answer  and  concluded  with  a call  for  accountability  and  common 
sense.  Cuban  (1996)  had  a similar  response  and  couched  his  assessment  in  an  historical  context. 

First,  techno-reformers'  claims  for  what  new  machines  can  do  are  so  inflated  that  public  expectations 
continually  get  disappointed.  Overselling  has  been  (and  continues  to  be)  part  of  a familiar  American  cycle  of 
creating  a crisis,  naming  schools  or  teachers  as  a problem,  and  putting  forward  new  machines  (film,  television, 
computers)  as  the  best  solution.  Yet  each  technological  innovation  has  had,  at  best,  an  uneven  record  in  entering 
schools  and  classrooms.  Why? 
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Based  on  this  exploration  of  WWW  use  in  school,  several  findings  would  appear  to  have  policy 
implications  for  schools  using  or  making  plans  to  use  the  WWW  for  educational  purposes.  First,  while  students 
believe  the  WWW  to  be  a valuable  source  of  reliable  information,  their  use  of  the  WWW  suggests  other 
motivations.  Analysis  of  sites  visited  indicated  that  by  nearly  a two-to-one  margin  students  visited  sites  rated 
"unsuitable  for  academic  research"  versus  sites  rated  "suitable."  Seeking  out  "pleasurable  experience"  appeared  to 
win  out  over  "learning  information"  (Swanson,  1992)  when  students  were  given  access  to  the  WWW  within  the 
school  setting.  Furthermore,  the  types  of  sites  visited  most  frequently,  i.e.,  commercial  sites,  were  rated  as  having 
the  lowest  educational  value. 

Also  of  note  is  the  incongruity  between  students'  self-reported  use  of  the  WWW  and  the  uses  suggested  by 
the  analysis  of  sites  visited  by  students.  Either  students  falsely  reported  their  intentions  or  intervening  variables 
affected  the  process  of  searching  for  and  obtaining  relevant  information.  One  untested  hypothesis  to  emerge  from 
this  study  is  that  the  best  of  intentions  may  be  confounded  by  the  ease  with  which  students  can  access  a myriad  of 
competing  sites  that  vie  for  their  attention.  Another  possibility  is  that  the  students'  understanding  of  research  is  more 
broadly  defined  and  includes  looking  for  content  that  has  little  or  no  relationship  to  traditional  academic  pursuits. 

When  it  comes  time  to  evaluate  the  appropriateness  and  effectiveness  of  media  technology  in  the  schools 
media  effects  researchers  cannot  have  it  both  ways.  Either  media  effects  are  real  and  the  potential  benefit  of 
educational  media  must  be  balanced  by  constant  vigilance  against  access  to  WWW  sites  that  are  at  best  a distraction 
and  at  worst  a hindrance  to  the  educational  and  social  development  of  our  children.  Or,  media  effects  are  limited  and 
mediated  by  user  motives,  attitudes,  and  use  patterns,  and  any  potential  benefit  of  educational  media  in  the  schools  is 
contingent  on  the  proper  psychological  and  sociological  predictor  variables.  If  this  is  the  case,  attention  to  these 
factors  must  be  a top  priority  and  WWW  access  must  be  implemented  with  the  goal  of  creating  the  proper  climate 
for  learning  to  occur.  In  either  case  WWW  literacy  efforts-te aching  students  how  to  most  effectively  use  the  best 
resources  on  the  WWW-are  sorely  needed.  Since  picking  and  choosing  only  the  best  WWW  resources  for  students  is 
not  a viable  option,  giving  students  the  tools  to  make  wise  decisions  about  media  content  is  crucial. 

Endnotes 

1 According  to  Wartella  and  Reeves  (1985)  the  first  recorded  instance  of  concern  about  media's  effect  on  children 
was  Plato's  warning  about  storytellers  in  The  Republic. 

2 One  could  argue  that  Illich  (1970)  envisioned  the  WWW  as  an  educational  resource  long  before  its  time  when  he 
wrote,  "The  current  search  for  new  educational  funnels  must  be  reversed  into  the  search  for  their  institutional 
inverse:  educational  webs  which  heighten  the  opportunity  for  each  one  to  transform  each  moment  of  his  living  into 
one  of  learning,  sharing,  and  caring"  (pp.  xix-xx,  emphasis  in  the  original).  Illich  continued,  "We  need  research  on 
the  possible  use  of  technology  to  create  institutions  which  serve  personal,  creative,  and  autonomous  interaction  and 
the  emergence  of  values  which  cannot  be  substantially  controlled  by  technocrats.  We  need  counterfoil  research  to 
current  futurology"  (p.  2).  In  describing  an  alternative  to  school,  Illich  might  have  been  describing  the  modem 
listserv  or  chat  forum:  "The  most  radical  alternative  to  school  would  be  a network  or  service  which  gave  each  man 
[sic]the  same  opportunity  to  share  his  [sic]  current  concern  with  others  motivated  by  the  same  concern"  (p.  19).  The 
great  difference,  however,  between  what  Illich  envisioned  and  what  WWW  advocates  are  promoting  is  the  locus  of 
responsibility. 

3 For  the  full  study,  including  survey  instruments,  see  Ebersole  (1999). 

4 It  is  interesting  to  note  that  the  list  of  "uses"  is  similar  those  found  by  researchers  exploring  the  uses  and 
gratifications  of  "old"  media.  Information,  entertainment,  social  utility,  passing  the  time,  and  other  traditionally 
defined  uses  of  the  media  are  present  with  some  distinctions  made  possible  by  the  interactive  nature  of  this  new 
medium. 
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Responsive  Instructional  Design: 
Scaffolding  the  Adoption  and  Change  Process 

Peggy  A.  Ertmer 
Purdue  University 


According  to  the  most  recent  report  of  the  National  Center  for  Education  Statistics  (NCES,  2000),  99 
percent  of  all  public  school  teachers  now  have  access  to  computers  or  the  Internet  in  their  schools.  However,  despite 
this  growing  presence  of  technology,  teachers  all  over  the  country  continue  to  grapple  with  both  practical  and 
philosophical  problems  posed  by  the  integration  process.  Nearly  70  percent  of  teachers  still  report  not  feeling  well 
prepared  to  use  computers  and  the  Internet  in  their  teaching  (NCES).  According  to  the  1998  Technology  in 
Education  Report  (Market  Data  Retrieval),  only  7 percent  of  schools,  nationwide,  boast  a majority  of  teachers  at  an 
advanced  skill  level  (i.e.,  able  to  integrate  technology  into  the  curriculum).  Even  among  our  newest  teachers, 
instructional  use  is  not  as  high  as  might  be  expected.  Although  beginning  teachers  report  wanting  to  use  computers, 
they  often  find  it  difficult  to  do  so  when  so  much  of  their  time  is  consumed  by  the  routine  tasks  of  teaching  and 
managing  their  classrooms.  The  use  of  computers,  then,  becomes  an  extra  task  which,  although  seen  as  important,  is 
of  lower  priority  than  more  basic  activities  (Hruskocy,  1999;  Novak  & Knowles,  1991). 

Clearly,  the  growing  increase  in  teachers'  technical  skills  is  insufficient  to  guarantee  the  effective  use  of 
technology  in  the  classroom  (Carvin,  1999;  Marcinkiewicz,  1994).  Through  previous  research  efforts  (Ertmer, 
1999;  Ertmer,  Addison,  Lane,  Ross,  & Woods,  1999;  Ertmer  & Hruskocy,  1999;  Hruskocy,  Cennamo,  Ertmer,  & 
Johnson,  2000)  we  have  identified  a number  of  challenges  that  teachers  face  as  they  begin  to  integrate  technology 
within  their  curricula.  Besides  a host  of  technical  and  logistical  questions  (e.g.,  How  does  this  software  package 
work?  Where  and  when  should  we  use  computers?),  more  subtle  issues,  related  to  teachers'  pedagogical  visions  and 
beliefs,  as  well  as  their  perceived  confidence  for  using  technology,  are  also  known  to  impede  meaningful  classroom 
use. 

Assuming  that  most  designers  and  staff  developers  are  fairly  well-prepared  to  provide  technical  skills 
training  for  teachers  who  want  it  and  are  prepared  to  learn  from  it , this  paper  focuses  instead  on  how  designers 
might  work  with  teachers  who  either  don't  want  training  or  are  not  prepared  to  learn  from  it,  or  both!  This  paper, 
then,  emphasizes  the  role  that  teachers'  beliefs  play  in  the  adoption  and  change  process  and  specifically  discusses 
how  those  beliefs  might  be  addressed  through  teacher  development  efforts.  Elsewhere  (Ertmer,  1999)  I have 
described  how  teachers'  beliefs  may  function  as  second-order  barriers  to  change  (i.e.,  barriers  that  are  intrinsic  to 
teachers  and  that  challenge  fundamental  beliefs  about  current  practice).  Again,  I am  assuming  that  designers  are 
capable  of  helping  teachers  deal  with  first-order  barriers,  those  that  are  extrinsic  to  teachers  and  which  include  skills 
training  and  equipment  needs.  Even  if  these  assumptions  are  false,  second-order  barriers  are  still  likely  to  present 
the  biggest  challenges  to  those  charged  with  coaching/assisting  reluctant  teachers/adopters. 

Teacher  Beliefs 

In  summarizing  research  on  teachers'  beliefs,  Pajares  (1992)  noted  that  "there  is  a strong  relationship 
between  teachers'  educational  beliefs  and  their  planning,  instructional  decisions,  and  classroom  practices"  (p.  326). 
In  particular,  teachers'  beliefs  about  their  ability  to  use  computers  in  instruction  may  be  key,  given  the  role  self- 
efficacy  is  proposed  to  play  in  determining  behavior.  In  a recent  study  by  McKinney,  Sexton,  and  Meyerson  (1999), 
participants  with  lower  efficacy  beliefs  expressed  concerns  typical  of  those  in  an  early  stage  of  change  (self- 
concems)  while  those  with  higher  efficacy  had  concerns  that  were  more  characteristic  of  later  stages  of  change 
(impact-concerns). 

Self-efficacy  refers  to  personal  beliefs  about  one's  capability  to  learn  or  perform  actions  at  designated  levels 
(Bandura,  1997).  According  to  Bandura,  self-efficacy  is  based,  not  solely  on  the  level  of  skill  possessed  by  an 
individual,  but  on  judgments  about  what  can  be  done  with  current  skills.  As  such,  self-efficacy  is  thought  to  mediate 
the  relationship  between  skill  and  action.  Simply  put,  without  skill,  performance  isn't  possible;  yet  without  self- 
efficacy,  performance  may  not  be  attempted.  According  to  Bandura,  "beliefs  of  personal  efficacy  constitute  the  key 
factor  of  human  agency"  (p.  3).  Thus,  teachers  who  have  high  levels  of  efficacy  for  teaching  with  technology  are 
more  likely  to  participate  more  eagerly,  expend  more  effort,  and  persist  longer  on  technology-related  tasks  than 
teachers  who  have  low  levels  of  efficacy. 
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So  what  does  this  mean  to  designers  and  others  who  are  responsible  for  teacher  development?  How  can  we 
design  professional  development  experiences  that  address  teachers'  second-order  barriers,  or  more  specifically,  that 
build  teachers'  efficacy  for  using  computers  in  instruction? 

Addressing  Efficacy  Beliefs  through  Design  Efforts 

Consider  the  following  scenario:  A new  principal  has  just  assumed  leadership  of  the  local  high  school  and 
has  some  fairly  strong  ideas  about  how  technology  should  be  used  within  her  school.  The  high  school  building  is 
well-equipped,  having  all  the  necessary  resources.  The  principal  approaches  you,  as  an  instructional  designer,  to 
help  her  people  get  up  to  speed.  She  admits  that  many  of  her  staff  are  not  convinced  of  the  need  for  technology  in 
their  teaching,  but  believes  that  if  you  present  a few  good  reasons  why  technology  should  be  used,  they  will  jump  on 
board.  Where  do  you  begin? 

Researchers  in  the  area  of  self-efficacy  (c.fi,  Schunk,  2000)  describe  four  primary  sources  of  information 
that  can  influence  judgments  of  efficacy:  vicarious  experiences  (observing  models),  social  persuasion  ("I  know  you 
can  do  this!"),  physiological  indicators  (emotional  arousal,  relaxation),  and  personal  mastery  (successful  task 
completion).  However,  teachers  may  be  more  or  less  influenced  by  each  source  of  information  depending  on  a host 
of  other  factors— their  current  skill  level,  their  beliefs  about  teaching  and  learning,  their  attitudes  towards  computers 
in  general,  and  so  on.  For  example,  providing  models  of  exemplary  users  may  not  be  effective  for  teachers  whose 
confidence  and  skill  levels  are  at  the  low  end  of  the  continuum.  In  fact,  the  use  of  this  type  of  efficacy  information 
may  prove  detrimental  for  teachers  who  can't  imagine  ever  attaining  these  high  levels  of  use  (Snoeyink,  2000). 
Instead,  teachers  with  low  skill  and  confidence  levels  may  need  to  experience  a few  small-scale  successes  in  order  to 
establish  even  an  initial  level  of  efficacy. 

Acknowledging  that  teachers  in  any  given  school  are  likely  to  represent  a range  of  levels  of  confidence, 
skills,  and  teaching  beliefs,  designers  can  not  expect  to  implement  a one-size-fits-all  training  program.  Still,  within 
each  stage  of  development  (novice  - expert),  growth  in  one  area  is  likely  to  be  related  to  growth  in  other  areas.  Thus, 
by  increasing  confidence,  skill,  or  a change  in  beliefs,  we  may  be  able  to  support  teachers'  efforts  toward  adoption 
and  use.  Certainly,  growth  in  skill  and  growth  in  confidence  have  been  demonstrated  to  be  positively  correlated 
(Ertmer,  Evenbeck,  Cennamo,  & Lehman,  1994).  Furthermore,  there  is  some  evidence  to  support  the  idea  that 
pedagogical  beliefs  and  technology  use  are  also  related  (Becker  & Riel,  1999)  and  that  as  teachers  use  technology 
more  their  beliefs  become  more  constructivist-oriented  (Dwyer,  1996). 

Consider  the  following  teachers  who  are  likely  to  comprise  part  of  the  teaching  staff  at  the  local  high 

school: 

Teachers  with  low  skills  and  low  confidence.  Low  skills  and  low  confidence  typically  translate  into 
resistant  or  reluctant  teachers  who  see  no  reason  to  initiate  change  in  their  classrooms.  Trying  to  convince  these 
teachers  that  they  need  to  use  technology  is  pointless;  strategies  that  worked  to  convince  early  adopters  are  not  likely 
to  work  with  late  adopters.  According  to  Moore  (cited  in  McKinzie,  1999),  "crossing  the  chasm  between  these 
groups  (early  and  late  adopters)  requires  a mammoth  campaign  that  includes  special  attention  to  the  vastly  different 
needs,  perspectives,  and  demands  of  the  late  adopters."  Whereas  social  persuasion  may  be  a great  strategy  for 
convincing  early  adopters,  this  strategy  usually  won't  work  with  late  adopters. 

Neither  is  it  likely  that  teachers  at  this  level  will  be  swayed  through  vicarious  experiences,  or  observing 
successful  others.  Teachers  at  this  level  are  not  ready  to  embrace,  or  even  consider,  technology  integration  ideas. 
Although  many  educators  have  suggested  that  technology  training  efforts  should  focus  on  the  conceptual  and 
pedagogical  levels  (Carvin,  1999;  Clouse  & Alexander,  1997),  others  disagree  (Milone,  2000;  Snoeyink,  2000). 
Stapel  (cited  in  Milone)  explained,  "In  the  past,  I think  we  made  a mistake  by  rushing  too  quickly  into  the 
integration  phase  of  staff  development.  In  many  cases,  teachers  just  weren't  ready"  (p.  58). 

Of  the  four  techniques  recommended  for  increasing  self-efficacy,  personal  mastery  is  likely  to  be  the  most 
beneficial  for  novice  computer  users.  Experiencing  even  small  success  can  increase  confidence.  Furthermore, 
establishing  relevancy  may  convince  reluctance  users  to  "have  a go."  A designer  in  the  situation  described  earlier 
might  begin  with  the  teachers'  own  personal  or  instructional  needs  as  one  way  to  create  relevance.  Perhaps  the 
teacher  has  a nephew  or  a grandchild  who  is  having  a birthday— would  she  like  to  send  a virtual  bouquet?  What 
particular  classroom  issues  are  consuming  the  teacher's  time?  Could  technology  ease  some  of  the  load  by  allowing 
her  to  easily  modify  newsletters  or  student  progress  reports  or  calculate  weighted  grades?  Is  there  a particular 
concept  that  is  hard  for  her  students  to  grasp  that  could  be  demonstrated  through  a simulation?  The  idea  is  to  find 
out  what  the  teacher  needs  and  then  collaboratively  develop  a plan  that  efficiently  meets  those  needs.  Similar  to  the 
idea  of  creating  an  individual  educational  plan  (IEP)  for  each  student,  a designer  might  consider  co-creating  an 
individual  teaching/technology  plan  (ITP)  with  each  teacher.  As  with  any  good  instructional  design,  designers  are 
advised  to  start  with  what  teachers  know  and  then  use  strategies  that  meet  their  unique  learning  needs. 


Teachers  with  medium  skills  and  medium  confidence.  Teachers  at  this  level  have  typically  achieved  some 
level  of  competence  with  technology  but  may  not  have  found  the  desire  or  means  to  bring  these  tools  into  the 
classroom.  Because  they  teachers  have  at  least  an  initial  level  of  confidence,  they  may  be  at  a point  where  they  will 
benefit  from  observing  similar  others  who  have  achieved  higher  levels  of  use.  These  applications  no  longer  seem 
out  of  reach  for  these  teachers  and  may  now  serve  as  inspiration.  Training  emphases  and  methods  that  were 
appropriate  for  novice  technology  users  (step-by-step  procedures,  skill  building)  are  not  likely  to  be  as  effective  for 
intermediate  users.  Teachers  at  this  level  may  benefit  from  shadowing  similar  others,  do-it-yourself  manuals,  and 
collaborative  efforts.  While  skills  will  need  to  be  continually  refined  and  updated,  it's  not  as  important  to  focus 
development  efforts  strictly  on  skills;  skill  instruction  can  now  be  embedded  within  meaningful  projects  or  relevant 
tasks. 

Teachers  at  a medium  skill  level  are  probably  just  starting  to  imagine  some  specific  classroom  applications 
and  may  not  be  able  to  extrapolate  from  ideas  used  at  other  grade  levels  and  in  other  content  areas  (Snoeyink,  2000). 
Thus,  it  may  still  be  important  at  this  level  to  include  concrete  ideas  for  the  teacher's  grade  level  and  content  area.  A 
designer  charged  with  helping  teachers  at  this  level  will  need  to  have  a strong  working  knowledge  of  various 
curricular  standards,  or  have  ready  access  to  someone  who  can  help  translate  these  standards  into  workable 
technology  solutions.  Unfortunately,  this  is  not  nearly  as  easy  as  it  sounds. 

Responsive  Instructional  Design 

In  the  following  section,  I propose  a model  of  teacher  development  that  builds  on,  and  responds  to, 
teachers'  unique  needs,  specifically  those  with  lower  levels  of  skill  and  confidence.  I propose  that  this  model 
represents  a responsive  approach  to  instructional  design;  that  is,  training  needs  are  described  in  terms  of  users  rather 
than  instruction,  and  instructional  decisions  are  based  on  users'  goals  and  needs,  not  those  of  designers  or  instructors. 
Although  the  model  is  similar  to  the  ADDIE  approach,  each  step  requires  that  more  attention  be  paid  to  users' 
perceived  needs  and  goals.  This  suggests  a greater  need  for  both  communication  and  analysis  skills.  Steps  in  the 
model  are  described  below. 


Responsive  Instructional  Design 


Reveal  Current  Beliefs 


ioc 


> 


Refine  Beliefs  and 

n 

Propose  Strategies 

Building  Efficacy; 

L 

Scaffolding  Change 

Reflect  on  Changes 

Implement  Changes 

Figure  1.  A Model  for  Responsive  Instructional  Design 


Reveal.  Responsive  instructional  design  is  based  on  the  assumption  that  teachers  are  more  likely  to  embrace 
pedagogical  and  classroom  change  if  these  changes  address  the  real  issues  they  face  in  practice.  Teachers  are 
encouraged  to  reveal  the  goals  they  want  to  accomplish  in  their  classrooms,  the  barriers  that  hinder  their  work,  and 
the  instructional  and/or  administrative  concerns  they  have,  unrelated  to  technology . Teachers  also  are  asked  to 


reveal  their  beliefs  about  teaching  as  well  as  the  incentives  that  motivate  them  to  teach.  At  these  early  stages  of 
technology  adoption  and  use,  the  focus  is  not  on  technology  skills  or  needs.  Teachers  are  encouraged  to  consider 
their  needs  as  both  teachers  and  learners  prior  to  considering  their  needs  as  technology  users.  Teachers  and 
designers  work  collaboratively  to  develop  an  individual  teacher  profile,  followed  by  the  development  of  an 
individual  teaching/technology  plan  (ITP)  in  the  next  step.  Thus,  an  individual  teacher  profile  might  include 
information  about  teachers'  1)  classroom  practices,  2)  classroom  context,  3)  perceived  issues  and  barriers,  4)  beliefs 
about  teaching,  5)  motivation  for  teaching,  as  well  as  6)  preferred  ways  of  learning  and  teaching.  By  beginning  with 
teachers'  perceived  needs,  we  remove  the  focus  from  the  innovation  and  place  it  instead  on  teaching  practices  and 
the  important  issues  teachers  face. 

Propose . During  this  step,  teachers  and  designers  co-develop  an  individual  teaching/technology  plan  by 
considering  various  means  for  meeting  specific  needs  identified  in  the  teacher  profile.  After  reviewing  the  issues 
teachers  face,  designers  help  translate  these  important  questions  into  technology-based  learning  opportunities. 
Depending  on  the  type  of  support  teachers  request  or  require,  different  types  of  support  should  be  offered  (one-on- 
one  consulting,  just-in-time  training,  formal  classroom  training,  peer  collaboration  and  observation,  etc.).  Some 
strategies  will  work  more  readily  and  be  more  appealing  than  others,  depending,  at  least  to  some  extent,  on  the 
barriers  teachers  describe.  Different  barriers  (e.g.,  lack  of  confidence  vs.  lack  of  support)  typically  suggest  the  use 
of  different  strategies.  For  example,  if  teachers  mention  not  yet  feeling  comfortable  with  technology,  they  probably 
are  not  ready  to  begin  using  technology  in  the  classroom.  Instead,  they  need  to  increase  their  personal  comfort 
through  increased  individual  and  personal  use.  By  acknowledging  and  helping  teachers  work  through  specific  first- 
and  second-order  barriers,  we  help  them  identify  strategies  that  work  for  them  and  simultaneously  build  confidence 
in  their  ability  to  address  future  barriers. 

Implement . As  teachers'  test  their  ideas  in  their  classrooms,  they  experience  first-hand  what  works  and  what 
doesn't.  As  Maddin  (1997)  emphasized,  "the  real  learning  begins  in  your  classroom"  (p.  56).  Information  obtained 
through  direct  experience  is  one  of  the  most  powerful  means  to  shape  future  practice.  Because  self-efficacy  is  a 
fluid  construct,  it  changes  with  new  experiences.  While  early  success  can  raise  efficacy,  early  failures  may  lower  it. 
For  this  reason,  it  is  probably  important  that  reluctant  teachers  experience  as  much  success  as  possible  during  their 
first  few  attempts  to  use  technology  within  the  classroom.  Additionally,  teachers  should  set  realistic  goals  for 
themselves  since  they  will  measure  their  success  by  how  closely  they  meet  the  goals  they  have  set  (Pintrich  & 
Schunk,  1996).  It  is  not  critical  that  reluctant  teachers  implement  a highly  sophisticated  lesson  with  lots  of  bells  and 
whistles.  What's  most  important  is  that  they  are  successful.  Risk  and  surprise  need  to  be  eliminated,  or  at  least 
greatly  controlled. 

Reflect.  Kagan  (1992)  explained  that  changes  in  teachers'  beliefs  are  rarely  the  result  of  reading  and 
applying  research  findings.  Teachers  base  most  of  their  ideas  on  their  own  and  others'  experiences.  In  order  to 
promote  professional  growth  in  novices,  Kagan  recommended  that  teachers'  awareness  of  their  own  beliefs  be  raised 
followed  by  experiences  that  challenge  those  beliefs  and  promote  integration  of  new  ideas  into  current  belief 
systems.  Such  reflection  initiates  the  revision  process.  After  implementing  new  ideas  or  tools  in  the  classroom,  the 
teacher  takes  time,  with  or  without  the  designer,  to  reflect  on  the  teaching/leaming  processes  and  outcomes 
achieved.  Teachers  consider  how  the  teaching  and  learning  that  occurred  compared  to  what  was  expected.  As 
teachers  realize  that  their  "ability  to  successfully  utilize  technology  has  increased,  they  are  motivated  to  attempt  to 
learn  more  about  technology,  it’s  uses,  and  benefits"  (George  & Camarata,  1996,  p.  51).  However,  as  with  most 
teaching  experiences,  there  are  usually  many  opportunities  for  improvement.  Teachers  should  be  encouraged  to 
focus  their  reflections  primarily  on  what  the  students  did  or  did  not  do  in  response  to  the  lesson.  Based  on  this 
information,  teachers  can  consider  what  changes  need  to  be  made  to  effect  the  types  of  student  performances  or 
levels  of  thinking  desired. 

Refine.  In  this  final  step,  teachers  are  encouraged  to  discuss  their  instructional  changes  with  others  and 
consider  the  overall  usefulness  and  effectiveness  of  the  changes  they  have  initiated.  Based  on  conversations  with 
others,  teachers  are  encouraged  to  outline  their  next  steps  for  development.  This  may  include  implementing  a 
revised  version  of  the  lesson,  adding  one  more  idea  to  the  lesson,  or  reading  relevant  literature  to  examine  what 
others  have  done.  Revisions  made  after  each  iteration  are  not  likely  to  be  substantial;  however,  continual 
refinements,  over  time,  can  add  up  to  noticeable  differences.  As  teachers  continue  to  converse  with  others  about 
how  they  addressed  a relevant  issue  in  their  classrooms,  as  well  as  the  results  they  obtained,  they  initiate,  in  effect, 
new  cycles  of  development. 
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Relationship  of  Model  to  Current  ID  Practices 

Designers  who  have  been  trained  to  design  programs/instruction  using  the  instructional  systems  approach 
already  have  the  basic  tools  they  need  to  work  with  teachers  and  students  in  a responsive  manner.  For  example, 
using  analysis  tools,  we  determine  teachers’  goals  for  their  students,  their  classrooms,  and  themselves.  We  identify 
the  specific  supporting  (or  limiting)  contextual  factors  in  the  environment  in  which  teachers  currently  apply  their 
skills.  As  part  of  a learner  analysis,  we  identify  teachers'  current  levels  of  skills  and  confidence.  Given  these  goals 
and  current  skills,  we  determine  the  gap  that  exists,  and  then  identify  specific  needs  that  must  be  met.  As  part  of  the 
design  and  development  process,  we  devise  instructional  strategies,  materials,  and  job  aids  that  will  meet  teachers 
where  they  are  and  start  to  move  them  forward  toward  the  goals  they  have  identified.  As  teachers  implement  new 
skills  within  their  classroom  practice,  we  have  the  opportunity  to  co-reflect  on  what  is  working  and  what  is  not,  and 
to  make  adjustments  (in  both  training  and  implementation)  to  better  achieve  the  intended  outcomes.  Ongoing 
evaluation  allows  teachers  and  designers  to  determine  the  effectiveness  of  selected  approaches  and  to  make  revisions 
that  bring  us  closer  to  our  mutual  goals. 

Even  though  the  design  steps  for  the  ADDIE  and  the  responsive  ID  models  are  similar,  designers  need  to  be 
aware  of  critical  differences,  including  the  different  starting  points  and  the  increased  involvement  of  the  end-users. 
To  summarize,  some  specific  suggestions  for  instructional  designers  who  intend  to  assist  teachers  with  their 
technology  efforts  include: 

1)  Ask  teachers  about  their  visions  and  goals  for  their  classrooms.  What  do  they  want  to  be  able  to  do? 
What  are  their  priorities  for  teaching  and  learning? 

2)  Listen  to  teachers'  specific  needs.  What  kind  of  barriers  do  they  encounter?  What  do  these  barriers 
suggest  about  their  readiness  for  technology  use  and  the  strategies  that  may  be  needed  to  assist  them  at 
different  levels? 

3)  Co-create  flexible  solutions.  Be  prepared  to  meet  changing  needs.  Be  sure  that  problems  and  solutions 
are  rooted  in  teachers'  work.  Use  the  reveal-reflect-refine  cycle  described  above. 

4)  Adapt  innovations  to  fit  teachers'  needs.  Teachers  are  great  adapters— help  them  locate  and  select 
programs  or  software  that  will  work  for  them  and  their  particular  set  of  students. 

Conclusion 

A number  of  researchers  have  suggested  that  training  strategies  should  be  varied  to  meet  the  needs  of 
teachers  at  different  levels  of  technology  use  (Dwyer,  Ringstaff,  & Sandholtz,  1991;  Sherry,  Billig,  Tavalin,  & 
Gibson,  2000).  Given  the  long-term  nature  of  the  integration/adoption  process,  it  is  recommended  that  staff 
developers/instructional  designers  meet  implementation  needs  in  a responsive  fashion— that  is,  through  "iterative 
interventions"  (Frame,  1991)  that  meet  and  challenge  individual  teachers  at  their  current  levels  of  use.  As  teachers 
face  changing  needs,  the  strategies  designed  to  meet  them  must  also  change.  Furthermore,  different  strategies  are 
likely  to  be  more  or  less  effective  for  people  with  different  levels  of  efficacy.  It  is  important  not  to  discourage 
teachers  who  have  low  levels  of  confidence  by  surrounding  them  with  others  who  are  much  more  experienced  and 
confident.  By  designing  development  programs  that  start  with  the  concerns  that  these  teachers  have,  and  helping 
them  experience  some  initial  success  in  solving  their  own  problems,  we  have  a better  chance  of  making  headway  in 
the  adoption  process. 


References 

Bandura,  A.  (1997).  Self-efficacy:  The  exercise  of  control.  New  York:  Freeman. 

Becker,  H.  J.,  & Riel,  M.  M.  (1999).  Teacher  professionalism,  school-work  culture  and  the  emergence  of 
constructivist-compatible  pedagogies.  University  of  California,  Irvine:  Center  for  Research  on  Information 

Technology  and  Organizations.  Available  online  at:  http://www.crito.uci.edu/tlc/findings/special_report2/index.htm 
on  October  19,  1999. 

Carvin,  A.  (1999).  Technology  professional  development  for  teachers:  Overcoming  a pedagogical  digital 
divide.  The  Digital  Beat.  1(16),  1-5.  Available  online  at:  http://www.benton.org/DigitalBeat/db093099.html  on 
October  5,  2000. 


119 


128 


Clouse  R.  W.,  & Alexander,  E.  (1997).  Classrooms  of  the  21st  century:  Teacher  competence,  confidence 
and  collaboration.  Journal  of  Educational  Technology  Systems  26.  97-1 1 1 . 

Dwyer,  D.  C.  (1996).  The  imperative  to  change  our  schools.  In  C.  Fisher,  D.  C.  Dwyer,  & K.  Yocam 
(Eds.),  Education  and  technology:  Reflections  on  computing  in  classrooms  (pp.  15-33).  San  Francisco:  Jossey 
Bass. 

Dwyer,  D.  C.,  Ringstaff,  C.,  & Sandholtz,  J.  (1991).  Changes  in  teachers'  beliefs  and  practices  in 
technology-rich  classrooms.  Educational  Leadership.  48(8).  45-54. 

Ertmer,  P.  A.  (1999).  Addressing  first-  and  second-order  barriers  to  change:  Strategies  for  technology 
integration.  Educational  Technology  Research  and  Development.  47(4),  47-6 1 . 

Ertmer,  P.  A.,  Addison,  P.,  Lane,  M.,  Ross,  E.,  & Woods,  D (1999).  Examining  teachers'  beliefs  about  the 
role  of  technology  in  the  elementary  classroom.  Journal  of  Research  on  Computing  in  Education.  32,  54-72. 

Ertmer,  P.  A.,  Evenbeck,  E.,  Cennamo,  K.  S.,  & Lehman,  J.  D.  (1994).  Enhancing  self-efficacy  for 
computer  technologies  through  the  use  of  positive  classroom  experiences.  Educational  Technology  Research  and 
Development.  42(3).  45-62. 

Ertmer,  P.  A.,  & Hruskocy,  C.  (1999).  Impacts  of  a university-elementary  school  partnership  designed  to 
support  technology  integration.  Educational  Technology  Research  and  Development.  47(  1 ),  8 1 -96. 

Frame,  J.  A.  (1991).  Moving  toward  a staff  development  model  for  computer  learning.  Dissertation 
Abstracts  International.  52/03,  756A. 

George,  G.,  & Camarata,  M.  R.  (1996).  Managing  instructor  cyberanxiety:  The  role  of  self-efficacy  in 
decreasing  resistance  to  change.  Educational  Technology.  36(4).  49-54. 

Hruskocy,  C.  (1999).  Student  trainers  as  resource  technologists  (Project  START):  A study  of  classroom 
interactions  and  use  of  student  trainers.  Unpublished  doctoral  dissertation,  Purdue  University,  West  Lafayette,  IN. 

Hruskocy,  C.,  Cennamo,  K.  S.,  Ertmer,  P.  A.,  & Johnson,  T.  (2000).  Creating  a community  of  technology 
users:  Students  become  technology  experts  for  teachers  and  peers.  Journal  of  Technology  and  Teacher  Education. 
8(l),69-84. 

Kagan,  D.  M.  (1992).  Implications  of  research  on  teacher  belief.  Educational  Psychologist.  27(1).  65-90. 

Maddin,  E.  A.  (1997).  The  real  learning  begins  back  in  the  classroom:  On-the-job  training  and  support  for 
teachers  using  technology.  Educational  Technology  37(5).  56-69. 

Marcinkiewicz,  H.  R.  (1994).  Computers  and  teachers:  Factors  influencing  computers  use  in  the 

classroom.  Journal  of  Research  on  Computing  in  Education,  27(2).  220-237. 

Market  Data  Retrieval  (1998).  Technology  in  Education  Report.  Available  online  at: 
http://www.schooldata.com  on  September  4,  2000. 

McKinney,  M.,  Sexton,  T.,  & Meyerson,  M.  J.  (1999).  Validating  the  efficacy-based  change  model. 
Teaching  and  Teacher  Education,  39(5),  37-43. 

McKinzie,  J.  (1999).  Reaching  the  reluctant  teacher.  Available  online  at: 
http://fno.org/sum99/reluctant.html  on  September  4,  2000. 

Milone,  M.  (2000,  September).  Staff  development:  Getting  it  right  the  first  time.  Technology  and 

Learning,  21(2).  58-61. 

National  Center  for  Education  Statistics  (2000).  Public  school  teachers'  use  of  computers  and  the  Internet. 
Washington  DC:  U.  S.  Department  of  Education. 

Novak,  D.  I.,  & Knowles,  J.  G.  (1991).  Beginning  elementary  teachers'  use  of  computers  in  classroom 
instruction.  Action  in  Teacher  Education,  23(2).  43-5 1 . 

Pajares,  F.  (1992).  Self-efficacy  beliefs  in  academic  settings.  Review  of  Educational  Research.  66.  543- 

578. 

Pintrick,  P.  R.,  & Schunk,  D.  H.  (1996).  Motivation  in  education:  Theory,  research,  and  applications. 
Englewood  Cliffs,  NJ:  Merrill/Prentice  Hall. 

Schunk,  D.  H.  (2000).  Learning  theories:  An  educational  perspective  (3rd.  ed.).  Upper  Saddle  River,  NJ: 
Merrill/Prentice  Hall. 

Sherry,  L.,  Billig,  S.,  Tavalin,  F.,  and  Gibson,  D.  (2000).  New  insights  on  technology  adoption  in  schools. 
T.H.E.  Journal,  27(7),  43-46. 

Snoeyink,  R.  (2000).  Experienced  Teachers'  Perspectives  of  Learning  and  Using  Computer  Technology. 
Unpublished  doctoral  disseration. , Purdue  University,  West  Lafayette,  IN. 


120 


129 


Assessing  Adaptive  Instructional  Design  Tools  and  Methods  in  ADAPT17 


Deniz  Eseryel 

University  of  Bergen,  Norway 

J.  Michael  Spector 
Syracuse  University 


Abstract 

ADAPT17  is  a European  project  within  the  Information  Society  Technologies  programme  that  is  providing 
design  methods  and  tools  to  guide  a training  designer  according  to  the  latest  cognitive  science  and  standardisation 
principles.  Adapt17  addresses  users  in  two  significantly  different  domains  within  the  aviation  industry:  aircraft 
maintenance  training  and  air  traffic  control  training.  Because  these  two  subject  domains  are  quite  different  and 
those  who  perform  associated  tasks  also  differ  in  significant  ways,  it  is  our  hypothesis  that  outcomes  will  generalise 
to  other  complex  domains  and  users.  The  methods  integrated  into  ADAPT77  are  based  on  van  Merrienboer’s  (1997) 
four  component  instructional  design  model.  This  paper  reports  the  needs  assessment  procedures  and  outcomes 
associated  with  the  effort  and  indicates  how  they  are  informing  the  evaluation  plan. 

Introduction 

ADAPT11  is  a European  project  coordinated  by  the  Dutch  National  Aerospace  Laboratory  (NLR).  The 
effort  falls  with  the  European  Commission’s  Information  Society  Technologies  programme  (1ST;  see 
http://www.cordis.lu/ist/  for  additional  details)  and  includes  as  partners:  NLR,  the  Open  University  of  the 
Netherlands,  the  University  of  Bergen,  Seven  Mountains  AS,  EuroControl,  the  Swedish  Air  Traffic  Control 
Academy,  and  Piaggo  Aerospace.  The  project  began  in  January  2000  with  an  extensive  needs  assessment  of  training 
designers  within  the  aviation  industry  and  in  other  business  and  industry  sectors  that  involved  training  of  complex 
skills.  The  early  literature  review  and  training  requirements  analysis  indicated  that  the  most  appropriate  and  relevant 
methodology  was  the  four  component  instructional  design  model  (van  Merrienboer  & Dijkstra,  1997).  Before 
discussing  the  ADAPT11  needs  assessment  and  its  implications  for  implementation  and  evaluation,  we  would  first 
like  to  orient  the  reader  with  a brief  overview  of  the  European  Commission’s  research  programmes  and  1ST. 

The  European  Commission’s  Fifth  Framework  research  and  development  programme  extends  from  1998  through 
2002.  1ST  is  a single,  integrated  research  programme  within  that  framework  that  builds  on  the  convergence  of 
information  processing,  communications  and  media  technologies.  1ST  has  a budget  of  approximately  3.6  billion 
Euro,  and  is  managed  by  the  Information  Society  Directorate  General  of  the  European  Commission  (one  of  20 
commissioners  in  the  EC).  The  strategic  objective  of  the  1ST  is  to  realise  the  benefits  of  the  information  society  for 
Europe  both  by  accelerating  its  emergence  and  by  ensuring  that  the  needs  of  individuals  and  enterprises  are  met.  It  is 
managed  by  the  European  Commission,  with  the  assistance  of  the  1ST  Committee  consisting  of  representatives  of 
each  Member  and  Associated  State.  The  Commission  and  the  1ST  Committee  are  supported  in  their  work  by  an  1ST 
Advisory  Group  of  some  25  members  who  are  highly  experienced  in  this  field.  They  provide  independent  expert 
advice  concerning  the  content  of  the  1ST  work  programme,  which  includes  research  and  development. 

The  1ST  Programme  has  four  inter-related  specific  objectives  concerning  individuals,  enterprises,  multimedia 
technologies  and  enabling  technologies.  For  the  private  individual,  the  objective  is  to  meet  the  need  and  expectation 
of  high-quality  affordable  general  interest  services.  For  Europe’s  enterprises,  workers  and  consumers,  the  objective 
is  to  enable  individuals  and  organisations  to  innovate  and  be  more  effective  and  efficient  in  their  work,  thereby 
providing  the  basis  for  sustainable  growth  and  high  added-value  employment  while  also  improving  the  quality  of 
working  life.  In  the  sector  of  multimedia  content,  the  key  objective  is  to  confirm  Europe  as  a leading  force,  realising 
its  full  potential.  For  the  enabling  technologies,  which  are  the  foundations  of  the  information  society,  the  programme 
objective  is  to  drive  their  development,  enhance  their  applicability  and  accelerate  their  take-up  in  Europe. 

ADAPT11  addresses  all  four  objectives  but  is  tightly  focussed  on  instructional  design  methods  and  tools  as  a 
critical  enabling  technology.  The  next  section  contains  a brief  overview  of  the  effort,  the  approach  taken  and  the 
associated  work  plan. 
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Advanced  Design  Approach  for  Personalised  Training-Interactive  Tools  (ADAPT17) 

The  overall  goal  of  this  effort  is  to  create  and  validate  effective  training  design  methodologies,  based  on 
cognitive  science  and  the  integration  of  advanced  technologies,  so  that  Europe  can  better  meet  the  many  challenges 
of  the  information  society  of  the  21st  century.  The  aviation  industry  is  targeted  for  this  research  and  development 
effort  as  it  is  a key  industry  in  which  Europe  has  been  an  active  leader.  Two  quite  different  training  areas  (air  traffic 
control  and  aircraft  maintenance)  are  involves  so  as  to  insure  generalisability  of  the  methodology  and  tools  to  other 
subject  areas  and  industries. 

This  project  bridges  the  gap  between  complex  training  problems  and  new  technological  possibilities  by 
developing  and  validating  a training  design  framework  to  guide  the  use  of  state-of-the-art  cognitive  approaches 
within  advanced  training  systems.  A central  requirement  for  the  design  methodology  is  that  it  provide  for 
development  of  a personalised  training  and  education  trajectory.  This  includes  a specification  of  training  models  that 
account  for  and  adapt  to  trainees*  performances,  attention  strategies,  and  workload.  Another  important  requirement 
for  efficient  training  of  flexible  skills  for  complex  tasks  is  to  maintain  a close  and  natural  relationship  between 
knowledge  learning  and  skill  acquisition  activities  for  both  individuals  and  teams  (van  Merrienboer,  1997;  van 
Merrienboer  & Dijkstra,  1997). 

This  project  is  developing  a personalised  training  design  methodology,  providing  associated  design  tools 
for  efficient  realisation  of  that  methodology,  and  validating  the  methodology  in  different  training  domains  and  with 
different  trainee-levels.  The  validation  will  be  performed  in  the  context  of  aircraft  maintenance  and  air  traffic 
control.  Current  challenges  for  aviation  training  relate  to  the  increasing  complexity  of  dynamic  task  environments, 
increasing  time  constraints,  and  increasing  demands  for  cognitive  and  information-managing  tasks.  As  technology 
takes  over  or  automates  many  basic  tasks  and  adds  functionalities  to  operational  systems,  the  result  is  that  more 
demand  is  placed  on  humans  to  perform  higher  level  and  supervisory  tasks.  The  problems  within  aviation  training 
are  exemplary  of  highly  complex  and  automated  task  environments  that  require  flexible  skills  and  are  likely  to 
forecast  training  problems  in  other  professional  domains. 

The  project  proposal  is  for  36  months  beginning  January  2000,  with  an  estimated  total  of  175  person- 
months  required  for  completion.  The  effort  is  broken  into  these  9 work  packages: 

1.  Project  management 

2.  Literature  search 

3.  Design  needs  analysis 

4.  Specifications  for  method  and  tools 

5.  Development  of  method  and  tools 

6.  Validation  of  the  design  process 

7.  Validation  of  the  learning  process 

8.  Standardisation 

9.  Integration,  final  report  and  recommendations 

Validation  partners  from  the  aviation  industry  (Piaggio,  EuroControl,  and  the  Swedish  Air  Traffic  Control 
Academy)  will  directly  benefit  from  the  training  design  methodology  and  associated  tools.  These  partners  provide  a 
real-world  setting  and  associated  support  for  testing  the  design,  development,  and  validation  of  the  design 
methodology  and  training  developed  according  to  that  methodology.  The  design  process  is  user-centered  and 
involves  the  validation  partners  throughout  the  process.  The  process  is  iterative  so  that  initial  designs,  methods,  and 
tools  can  be  refined  in  close  association  with  the  requirements  and  needs  of  the  targeted  industry  users  and 
beneficiaries.  At  the  end  of  the  effort,  the  validation  partners  will  have  in  place  validated  training  modules  as  well  as 
a cadre  of  training  design  specialists  knowledgeable  and  skilled  in  the  use  of  the  newly  developed  design 
methodology  and  tools. 

Training  Design  for  Complex  Cognitive  Skills 

One  of  the  major  aims  of  ADAPT17  is  to  create  a harmonised  training-design  tool  for  dealing  with  tasks 
requiring  complex  cognitive  skills,  which  can  be  defined  as  “skills  for  which  the  learner  must  invest  considerable 
time  and  effort  to  acquire  an  acceptable  mastery  level  and  for  which  qualitative  differences  in  performance  exist 
between  novices  and  experts”  (van  Merrienboer  & Dijkstra,  1997,  p.427).  Training  of  complex  cognitive  skills  are 
becoming  increasingly  important  in  today’s  society  where  routine  tasks  (recurrent  skills)  are  mostly  automated  by 
machines.  As  a consequence,  complex  cognitive  tasks,  which  cannot  be  taken  over  by  machines,  form  the  basis  of 
training  needs  in  industry.  Examples  of  such  complex  cognitive  skills  are  air  traffic  control  skills,  fault-management 
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skills,  and  computer-programming  skills.  One  of  the  main  characteristics  of  such  skills  is  that  their  acquisition  is  a 
lengthy  and  effortful  process.  In  other  words,  they  are  hard  to  learn.  There  are  many  constituent  skills  involved  and 
at  least  some  of  those  constituent  skills  involve  problem  solving  and  qualitative  reasoning,  requiring  a deep 
understanding  of  systems  and  processes.  In  addition,  the  ability  to  integrate  and  coordinate  the  constituent  skills 
involved  is  critical  to  reaching  acceptable  performance.  Traditional  instructional  design  models  (e.g.,  Dick  and 
Carey,  1996;  Gagne,  Briggs,  & Wager,  1 992)  are  particularly  weak  when  it  comes  to  the  design  of  training  programs 
for  the  multidimensional,  complex  learning  required  in  highly  demanding  technical  domains  (Schneider,  1985). 
Other  alternative  models,  while  useful  at  a high  level,  are  typically  too  generic  and  fail  to  provide  specific  guidelines 
(e.g.,  Gustafson  & Branch,  1997;  Tennyson,  1995)  or  there  haven’t  been  validated  in  the  aviation  industry  (e.g., 
Kieras  & Poison,  1985).  Therefore,  the  ADAPT17  methodology  is  based  on  van  Merrienboer’s  Four-Component 
Instructional  Design  (4C/ID)  model.  The  4C/ID  model  involves  a training  design  method  that  has  been  empirically 
validated  in  the  aviation  industry  (van  Blanken  & van  der  Pal,  1999).  This  method  is  scenario-based  and  makes  the 
acquisition  of  complex  cognitive  technical  skills  the  primary  target  and  foundation  of  the  training  analysis  and 
design  method. 
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Figure  J.  Overview  of  the  4C/ID-model,  adapted  from  van  Merrienboer  (1 997). 

The  model’s  systematic  approach  to  the  development  of  training  is  represented  in  Figure  1.  The  model 
provides  guidelines  for  the  design  of  training  programs  in  highly  complex  domains  and  consists  of  four  layers:  (1) 
principled  skill  decomposition;  (2)  analysis  of  constituent  skills  and  related  knowledge;  (3)  selection  of  instructional 
methods;  and,  (4)  development  of  a learning  environment.  These  layers  prescribe  and  contextualise  the  activities  that 
instructional  designers  should  perform  in  order  to  produce  effective  training  for  complex  cognitive  skills.  Layers  1 
and  2 involve  the  analysis  of  a complex  cognitive  skill;  layers  3 and  4 involve  the  design  of  a training  strategy  and 
learning  environment  for  that  skill.  It  should  be  noted  that  although  the  layers  are  represented  in  a linear  order,  real 
world  applications  are  much  less  well-ordered  and  often  involve  iterations.  Tennyson  (1995)  also  emphasizes  this,  as 
well  as  the  associated  contextualised  activities  of  designers. 

Ideally,  the  application  of  the  model  starts  with  principled  skill  decomposition,  where  the  complex 
cognitive  skill  is  decomposed  into  a hierarchy  of  its  constituent  skills  and  categorized  according  to  their  desired  exit 
behaviours  and  related  types  of  learning.  This  phase  assumes  that  a proper  needs  analysis  has  been  conducted  and 
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the  general  goal  or  terminal  objective  of  the  training  has  been  identified.  Briefly  speaking,  in  the  principled  skill 
decomposition  phase,  this  general  goal  is  further  decomposed  into  a skills  hierarchy  that  reflects  the  more  specific 
constituent  skills  that  enable  the  performance  of  the  whole  skill.  The  general  idea  of  such  a skill  hierarchy  is  that 
“learning  the  constituent  skills  lower  in  the  hierarchy  might  facilitate  the  learning  or  performance  of  the  skills  higher 
in  the  hierarchy”  (van  Merrienboer,  1997,  p.86).  The  creation  of  hierarchies  has  been  a long-standing  practice  in 
many  instructional  models  (Dick  & Carey,  1996;  Gagne  et  al.,  1992),  but  this  particular  hierarchy  is  focussed 
entirely  on  complex  cognitive  skills. 


Macro  level  sequence 

cluster  1 cluster  2 ► cluster  3 


Figure  2.  Generic  example  of  a skills  hierarchy  and  its  division  into  skill  clusters 

For  many  tasks,  all  of  the  constituent  skills  of  the  complex  cognitive  skill  cannot  be  trained  simultaneously, 
so  the  skills  hierarchy  is  divided  into  clusters.  This  clustering  is  called  macro-level  sequencing.  The  clusters  may  be 
seen  as  “parts”  of  the  complex  cognitive  skill  and  each  cluster  contains  a fairly  large,  meaningful  set  of  interrelated 
constituent  skills.  Each  cluster  may  require  from  20  to  200  hours  of  training.  Also,  every  cluster  of  subskills  should 
represent  an  authentic  task  that  an  expert  would  perform  in  practice.  Figure  2 shows  an  example  of  a skills  hierarchy 
and  how  it  used  to  form  skill  clusters.  At  the  top  of  the  hierarchy  is  the  complete  complex  cognitive  skill.  The 
downward  links  represent  the  skills  that  are  prerequisite  for  mastery  of  the  skill  above.  For  example,  to  be  able  to 
perform  skill  2,  one  should  be  able  to  perform  subskills  2.1  and  2.2.  From  left  to  right,  a temporal  order  is  depicted. 
For  example,  for  the  complete  performance  of  skill  2,  one  first  performs  skill  2.1  and  then  skill  2.2.  Skills  on  the 
same  level  that  are  connected  with  a double  arrow  are  performed  concurrently.  In  the  example,  a backward  chaining 
approach  is  applied  to  order  the  skills  clusters.  This  means  that  the  cluster  of  skills  that  is  trained  first  is  the  cluster 
that  is  usually  performed  last  when  an  expert  performs  the  complete  complex  skill. 

Once  the  constituent  skills  are  described  along  with  performance  objectives,  categorized  (recurrent,  non- 
recurrent), and  sequenced,  the  next  layer  in  the  model  deals  with  the  analysis  of  those  identified  constituent  skills, 
relationships  between  them,  and  knowledge  structures  underlying  the  performance  of  the  complex  cognitive  skill 
and  its  constituent  skills.  This  analysis  primarily  refers  to  the  procedures  or  rules  that  underlie  expert  knowledge. 
Briefly,  in  this  phase  four  different  analysis  takes  place:  (1)  analysis  of  recurrent  constituent  skills  as  strong 
methods;  (2)  analysis  of  non-recurrent  constituent  skills  as  weak  methods;  (3)  analysis  of  prerequisite  knowledge  to 
perform  recurrent  skills;  and,  (4)  analysis  of  supportive  knowledge  to  perform  non-recurrent  constituent  skills. 
Different  task  analytical  techniques  are  prescribed  by  the  model  for  each  of  these  analyses  (van  Merrienboer,  1997). 
It  is  important  to  note  that  this  layer  translates  into  meso-level  sequencing  which  specifies  the  order  in  which  case 
types  (i.e.,  categories  of  problems  or  worked-out  examples)  should  be  included  in  the  training  sequence  for  the  skill 
clusters  defined  during  macro-level  sequencing.  In  other  words,  this  sequencing  provides  a first  global  blueprint  for 
the  contents  of  a training  program,  providing  a sound  basis  for  the  subsequent  design  of  a learning  environment.  The 
outcomes  of  the  analysis  conducted  in  this  second  layer  are  heuristic  methods  - systematic  approaches  to  problem 
solving  (SAP  charts).  These  charts  that  indicate:  the  sequence  of  non-recurrent  skills;  the  supportive  knowledge 
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involved;  rules  and  procedures  for  performing  recurrent  skills;  prerequisite  knowledge  required  to  be  able  to  perform 
procedures  and  apply  rules;  and  sets  of  case  types  in  a progression  of  simple  to  complex  versions  of  the  whole  skill 
within  the  cluster. 

After  analysing  the  complex  cognitive  skill  and  each  of  its  constituent  skills,  the  training  strategy  should  be 
composed.  This  coincides  with  the  third  layer  in  the  model,  where  instructional  methods  for  the  design  of  whole-task 
and  part-task  practice  as  well  as  for  different  types  of  information  presentation  before  and  during  practice  are 
specified.  Again  four  different  activities  take  place  depending  on  the  category  of  the  constituent  skills  at  hand: 
whole-task  practice,  supportive  information,  prerequisite  information,  and  part-task  practice.  The  design  of  whole- 
task  practice,  in  this  layer,  mainly  refers  to  specifications  of  examples  (problems  and  their  solutions)  and  problems 
(incomplete  examples)  that  the  learners  will  be  confronted  with  during  training.  This  process  should  aim  at 
promoting  “a  rapid  development  of  highly  situation-specific,  automated  rules  by  knowledge  compilation. 
Instructional  methods  that  are  suitable  for  to  reach  this  goal  are  often  associated  with  repeated  imitation  and  drill; 
learners  are  invited  to  mechanically  and  consistently  repeat  performance”  (van  Merrienboer  & Dijkstra,  1997,  p. 
437).  The  key  learning  process  to  be  promoted  is  induction  - mindful  abstractions  of  cognitive  schemata  that  are 
useful  to  solving  problems  in  the  domain  of  interest.  This  is  based  on  the  premise  that  giving  learners  examples 
helps  them  construct  mental  frameworks  that  they  can  use  when  encountering  similar  problems  in  the  future.  The 
design  of  whole-task  practice  results  in  the  first  blueprint  of  the  training  program,  describing  what  the  learners  will 
be  required  to  do  during  the  training.  Figure  3 depicts  the  basic  blueprint  of  a training  program  resulting  from 
macro-level,  meso-level,  and  micro-level  sequencing.  From  the  viewpoint  of  the  designer,  it  is  basically  a hierarchy 
with  the  complex  skill  at  the  top  followed  by  three  sequencing  levels:  (1)  skill  clusters;  (2)  case  types;  and,  (3) 
specific  whole-task  problems.  This  basic  skeleton  serves  as  the  backbone  for  other  design  activities  where  additional 
practice  for  recurrent  constituent  skills  and  information  is  presented  in  order  to  support  the  acquisition  of  either  non- 
recurrent or  recurrent  constituent  skills.  In  other  words,  the  training  program  blueprint  is  a hierarchy  of: 

• skill  clusters  (“whole  tasks”  - macro-level  sequence); 

• case  types  (the  simple-to-complex  cases  of  the  whole  task  - meso-level  sequence);  and, 

• specific  problems  (the  problems  or  worked-out  examples  in  each  type  of  case  and  in  each  skill  cluster  - micro- 
level sequence). 

This  basic  blueprint  as  presented  in  Figure  3,  is  then  further  elaborated  to  reach  a complete  description  of  the 
training  program  with  the.other  activities  in  the  third  layer,  which  provide: 

• additional  part-task  practice  that  may  be  necessary  to  reach  the  required  exit  behavior  for  particular  recurrent 
constituent  skills  (part-task  practice); 

• information  that  is  prerequisite  to  the  performance  of  recurrent  constituent  skills,  as  performed  either  in  a 
whole-task  or  part-task  context  (prerequisite  info/JIT  presentation);  and, 

• information  that  may  be  helpful  to  the  performance  of  non-recurrent  constituent  skills,  which  are  only 
performed  in  a whole-task  context  (supportive  info  for  elaboration  and  understanding). 

The  model  further  provides  detailed  guidelines  for  instructional  strategies  and  tactics  for  the  presentation  of 
information  for  each  of  the  four  components:  whole-task  practice,  part-task  practice,  just-in-time  information,  and 
supportive  and  strategic  information  that  need  to  be  elaborated  by  the  learners  (van  Merrienboer,  1997).  Once, 
appropriate  strategies  and  tactics  are  selected  in  the  third  layer,  the  designer  moves  to  the  fourth  layer  to  compose  a 
training  strategy  containing  the  selected  instructional  methods;  this  process  results  in  a detailed  blueprint  for  the 
learning  environment.  This  layer  prepares  the  designer  for  the  transition  from  the  design  phase  to  the  development 
phase.  Since  4C/ID  is  an  instructional  design  model,  not  an  instructional  development  model,  it  does  not  provide 
detailed  guidelines  for  the  development  of  instructional  materials.  The  model  should  be  integrated  with  and 
embedded  in  an  instructional  development  model  during  its  implementation. 
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Complex  Cognitive  Skill 


Overall,  the  ADAPT  system  aims  at  this  integration.  The  result  be  a more  time-effective  instructional 
design  process  that  increases  the  possibilities  for  re-usability  of  intermediate  design  products.  Within  the  system,  the 
ADAPT  method  that  will  guide  the  design  phase  will  be  based  on  the  4C/ID  model  just  described  but  also  integrate 
models  of  workload  for  aviation  training  development.  In  order  to  reach  sufficient  levels  of  standardisation  and 
generality,  designers  will  be  supported  in  the  proper  and  efficient  use  of  the  methodology  by  means  of  an  interactive 
computer-based  tool  - automated  instructional  design  support  - ADAPT11.  One  of  the  innovative  aspects  of 
ADAPT17  is  that  it  will  assist  designers  and  design  teams  in  decomposing  and  analysing  a complex  skill  to-be-taught 
in  such  a way  that  alternative  learning  tasks  can  be  developed  that  vary  on  the  degree  of  cognitive  load  imposed  on 
the  learner.  The  ADAPT  method  will  address  the  whole  range  of  training  design  while  including  the  aspects  of 
‘intelligent  tutoring’  that  relate  to  cognitive  load  theory. 

Training  Designer’s  Needs 

Although  the  ADAPT  methodology  will  be  mainly  based  on  the  4C/ID  model  it  will  be  different  from  the 
4C/ID  method  in  two  ways.  First,  it  will  be  tailored  to  the  needs  of  actual  designers  or  design  teams  in  a variety  of 
training  domains,  economical  sectors,  and  company  sizes.  Second,  it  will  be  extended  to  become  a personalised 
training  approach.  In  order  to  achieve  the  former,  current  practice  in  designing  training  for  complex  cognitive  skills 
had  to  be  identified.  The  project  team  realised  this  through  three  different  data  sources.  First,  the  literature  on 
training  designer’s  needs  was  reviewed.  Training  designer’s  needs  are  defined  as  the  requirements  that  have  to  be 
met  in  order  to  change  and  improve  current  design  practice.  These  requirements  refer  primarily  to  instructional 
design  practitioner’s  perceived  needs  for  support  to  use  and  apply  instructional  design  theories  and  models.  In 
addition  to  the  literature  review,  designer’s  needs  were  identified  by  means  of  questionnaires  and  interviews.  The 
design  of  the  questionnaire  was  a collective  effort  based  on  focus  groups  with  project  and  subject  experts.  It 
consisted  of  the  following  sections: 

1 . Background  information  (information  on  the  company  and  the  responding  designer); 

2.  Outcomes  of  training  (the  tasks,  skills,  knowledge  and  attitudes  for  which  training  is  designed); 

3.  Training  design  method  (the  products  of  the  training  design  process,  both  intermediate  and  final); 

4.  Instructional  design  tools  (the  tools  that  are  used  to  design  training);  and, 

5.  Desired  tool  characteristics  (the  ideal  characteristics  and  functionality  that  respondents  would  like  to 
have). 

A total  of  150  questionnaires  were  sent  out  by  targeted  e-mail,  and  18  were  returned.  The  respondents  came  from 
different  sectors:  education,  transport,  business,  and  manufacturing.  In  addition  to  the  questionnaires,  extensive 
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interviews  with  training  designers  were  also  conducted  in  order  to  gather  more  detailed  information  on  current 
training  design  practices,  the  problems  that  are  encountered  and  the  desired  support  and  specific  needs  during  the 
training  design  process.  The  interview  consisted  of  the  following  sections: 

1 . General  information  (information  on  the  company  and  the  interviewee); 

2.  Training  design  process  (the  training  design  process  in  general  and  the  products  of  the  training  design 
process  - both  intermediate  and  final  products); 

3.  Training  design  characteristics  (the  content  of  the  training,  the  instructional  principles  used,  etc.); 

4.  Co-operative  design  and  design  teams  (the  structure  of  a design  team,  the  disciplines  involved  in  a typical 
design  team,  the  way  in  which  a typical  design  team  works,  etc.);  and, 

5.  Instructional  design  tools  (the  tools  that  are  used  and  ideal  characteristics  of  an  instructional  design  tool). 

Each  interview  took  at  least  three  hours.  The  target  group  was  training  designers  who  are  actively  involved 
in  the  design  of  training  for  complex  domains.  The  interviews  were  conducted  by  two  people,  one  to  provide  direct 
eye  contact  and  ask  questions  and  one  to  record  answers  and  prompt  for  specific  questions  if  needed.  A structured 
interview  instrument  guided  the  interview  process  to  insure  that  all  respondents  were  asked  similar  questions.  In 
total  1 1 interviews  were  conducted  in  the  following  sectors:  1 naval  college,  3 air  traffic  control  centres,  2 aircraft 
maintenance  organisations,  1 telecommunications  organisation,  1 information  technology  company,  1 training 
consultancy  organisation  and  1 distance  education  company.  Unlike  the  questionnaires  that  produced  data  in  one 
format,  the  more  in-depth  interviews  were  intended  to  provide  detailed  and  rich  information  about  company  design 
practices.  As  interviewee  background  and  companies  differ  substantially,  the  interviews  were  loosely  structured  in 
the  sense  that  questions  were  open  and  no  pre-defined  answer  categories  were  supplied.  This  was  intended  to 
stimulate  the  interviewer  towards  more  natural,  personal  and  company-specific  responses,  and  this  aspect  was  a 
complete  success.  Nevertheless,  some  structure  was  provided  for  the  interviewer  by  a list  of  questions  and  themes  to 
be  addressed.  Interviewer  instructions  were  also  provided  since  there  were  multiple  interviewing  pairs.  This  was 
intended  to  increase  the  compatibility  between  the  interviews  and  over  the  variety  of  questions  asked.  The 
interviews  not  only  yielded  more  detailed  information,  but,  more  importantly,  they  yielded  an  impression  or  feeling 
of  current  design  practice,  which  did  not  emerge  from  the  questionnaires. 

The  emphases  during  these  data  gathering  efforts  was  on  the  difference  between  current  and  ideal  training 
design  practice,  on  the  problems  experienced  during  the  design  process,  and  on  the  ideal  characteristics  of  an 
instructional  design  tool.  In  analysing  the  results,  our  interest  was  on  the  common  themes  that  arose.  In  general,  it 
can  be  concluded  that  current  training  design  practice  differs  from  that  what  is  described  in  most  instructional  design 
models  (e.g.,  Gustafson  & Branch,  1997).  Activities  are  carried  out  selectively  and  often  partly  and  are  characterised 
by  implicit  and  intuitive  methods  as  suggested  by  Tennyson  (1995).  The  analysis  phase  is  not  carried  out  explicitly 
and  in  detail,  mostly  due  to  constraints  involving  time  and  resources.  The  design  phase  appears  often  to  be  combined 
with  the  development  phase  and  often  happens  implicitly  as  part  of  the  development  and  implementation  of  training. 

Looking  at  the  background  of  training  designers,  most  designers  have  an  operational  background  and  are 
not  specifically  trained  or  educated  in  instructional  design  or  training  systems.  Specific  instructional  design  models 
or  principles  are  rarely  explicitly  used  by  designers  with  an  operational  background.  However,  many  basic  ideas  and 
some  instructional  design  principles  seem  to  be  intuitively  understood  and  used.  These  principles  are  not  explicitly 
or  systematically  applied  to  a training  plan  or  design  blueprint. 

In  general,  training  is  not  designed  in  formally  structured  design  teams.  The  most  important  actors  in  a 
team  are  subject  matter  experts  and  instructors,  who  typically  design  the  training.  Collaborative  design  does  not 
seem  to  take  place  explicitly. 

In  answering  the  survey  questionnaires  and  extended  interviews,  respondents  indicated  that  the  most 
significant  and  desired  characteristics  of  an  ideal  instructional  design  tool  are  related  to: 

• targeting  actual  designers  with  an  operational  background; 

• using  non-academic  language; 

• providing  practical  worked-out  examples; 

• supporting  an  explicit,  structured  and  systematic  design  process  (especially  for  the  analysis,  design  and 
evaluation  phases); 

• providing  guidance  in  bridging  the  gap  between  current  and  ideal  design  practice; 

• addressing  the  relational  aspect  during  the  training  design  process  (setting  up  teams,  involving  users, 
teamwork,  communication,  etc.); 
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• providing  support/guidance  in  applying  educational  design  principles; 

• providing  methods  for  file  management  and  version  control; 

• linking  together  the  different  training  design  products; 

• making  information  easily  re-usable  and  retrievable;  and, 

• allowing  different  degrees  of  freedom  (e.g.,  from  structured  to  non-sequential  design). 

The  interest  in  structure,  reusability,  and  relevant  examples  is  consistent  with  the  literature  search 
conducted  for  this  project  and  with  the  experience  of  project  experts.  Some  respondents  questioned  the  merit  of  such 
a tool  as  opposed  to  current  design  practice;  these  respondents  are  most  probably  the  most  experienced  of  those 
interviewed  and  are  justifiably  concerned  with  regard  to  the  a tool  constraining  the  application  of  their  advanced 
knowledge  and  skills.  ADAPT17  should  therefore  be  set  up  in  such  a way  that  designers  quickly  become  aware  of 
and  are  convinced  of  the  additional  value  of  ADAPT17,  while  allowing  advanced  users  to  integrate  their  own 
knowledge  and  skills.  In  addition,  designers  must  be  able  to  design  training  more  quickly  (increased  efficiency)  and 
to  design  better  training  (increased  effectiveness)  with  ADAPT17 

Assessment  within  ADAPTit 

The  project  is  addressing  two  additional  kinds  of  assessment  issues:  (1)  validating  the  tool  against  the 
theoretically-based  design  methodology  upon  which  it  is  based;  and,  (2)  demonstrating  the  effectiveness  of  the  tool 
and  the  method  in  terms  of  learning  outcomes.  The  outcomes  of  the  needs  assessment  phase  described  above  have 
led  to  specific  method  and  tool  requirements  that  are  being  integrated  into  the  first  prototype.  Field  tryouts  with 
users  in  real-world  settings  will  then  lead  to  a second  iteration  and  refinement  of  the  method  and  tool.  The  project 
will  assess  the  usability  and  utility  of  the  refined  method  and  tool  by  aviation  industry  practitioners.  In  addition,  the 
project  will  content  a quality  assessment  of  the  design  blueprints  produced  as  well  as  an  effectiveness  assessment  of 
the  training  developed  according  to  those  design  blueprints.  As  was  the  case  with  the  Experimental  Advanced 
Instructional  Design  Advisor  effort  (Spector,  1995),  ADAPT17  intends  to  improve  both  efficiency  and  effectiveness. 

Additional  requirements  for  ADAPT17  have  not  been  emphasised  in  this  paper  since  the  focus  has  been  on 
the  assessment  methods  and  their  implications.  However,  it  is  the  intent  to  include  a number  of  design  advisors, 
customised  to  the  appropriate  enterprise  setting.  In  the  aviation  industry,  safety  requirements  are  paramount  and 
these  must  be  emphasised  in  all  training,  both  for  maintenance  technicians  as  well  as  for  air  traffic  controllers. 
Moreover,  there  are  aviation  industry  standards  for  the  portability  of  training  set  by  the  Aviation  Industry 
Computing  Consortium  (AICC)  in  addition  to  the  IEEE  standards  being  developed  for  meta-data  in  technology- 
based  training.  Finally,  there  are  generic  instructional  design  standards  established  independently  by  the 
International  Board  of  Standards  for  Training,  Performance  and  Instruction  (ibstpi  - see  http://www.ibstpi.org)  that 
ADAPT17  is  committed  to  include.  The  overall  assessment  of  ADAPT17  will  include  an  external  evaluation  of  how 
well  the  method  and  tool  adhere  to  these  various  sets  of  standards. 

Conclusions 

While  there  have  been  efforts  in  the  area  of  automated  support  for  instructional  design  (Spector,  Poison,  & 
Muraida,  1993),  what  has  been  missing  in  this  area  are  sufficiently  elaborated  design  models  with  associated 
guidance  and  frameworks  that  are  appropriate  for  the  design  of  training  for  complex  cognitive  skills.  The  ADAPT17 
project  is  intended  to  fill  this  gap  in  the  set  of  methods  and  tools  available  to  instructional  designers.  The  outcome  of 
the  needs  assessment  phase  strongly  suggests  that  the  demand  for  such  an  integrated  method-tool  combination  is 
highly  desired  by  business  and  industry.  The  user-centred  process  adopted  for  this  project  insure  that  these  real 
world  needs  will  be  addressed  in  the  project  and  its  products.  The  assessment  of  European  designers  described  in 
this  paper  has  resulted  in  a set  of  design  requirements  that  will  also  be  used  for  evaluating  the  final  outcomes  of  the 
effort. 
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Abstract 

This  paper  presents  the  results  of  a student  evaluation  of  a web-based  distance  education  course  offered  by 
a major  university  in  the  midwestern  United  States . The  findings  of  this  study  found  that  some  students  experienced 
what  Hara  and  Kling  (2000)  call  distance  education  “ distress " and  that,  while  they  found  the  course  to  be  a 
satisfactory  educational  experience , their  online  interactions  were  judged  to  be  less  than  ideal 

Introduction 

Almost  all  students  from  elementary  to  higher  education  are  educated  in  a lecture  based  educational 
system.  The  communication  patterns  and  characteristics  of  face-to-face  lecture  based  environments  can  be  quite 
different  from  those  found  in  a distance  education  environment.  For  example,  in  a web-based  distance  education 
environment,  all  verbal  and  non-verbal  communication  cues,  traditionally  found  in  a face-to-face  environment, 
disappear.  In  this  new  environment,  supportive  teacher-student  interaction  and  student-student  networking  become 
very  important.  Although  many  argue  that  the  web  can  be  a good  environment  for  delivering  sound  educational 
experiences,  currently  there  is  very  little  solid  research  to  identify  key  issues  to  making  online  distance  education 
experience  a satisfying  one  for  students.  We  hope  that  this  research’s  results  will  contribute  to  the  efforts  to  close 
this  gap. 

Purpose  of  the  Study 

This  study  examined  some  of  the  factors  that  contribute  to  student  satisfaction  in  web-based  distance 
education  courses.  In  this  study,  by  examining  several  factors  based  on  Kirkpatrick’s  Four  Levels  of  Evaluation 
(Kirkpatrick,  1998),  the  researchers  explored  students’  responses  to  the  instructor,  activities  and  materials  of  a fully 
online,  graduate-level  course.  The  researchers  compared  their  findings  to  those  obtained  by  Hara  and  Kling  (2000), 
and  used  their  definition  of  “distress”  to  discuss  students’  negative  feedback  regarding  their  online  course 
experience. 

Statement  of  Significance 

As  stated  by  Keegan  (1996),  and  Reeves  in  Khan  (1997),  there  is  not  enough  and  quality  research  on 
several  important  aspects  of  web-based  distance  education.  This  research  effort  will  attempt  to  evaluate  student 
satisfaction  regarding  an  online  course  based  on  a common  structure  for  measuring  student  reaction  to  a course  or 
training  effort,  Kirkpatrick’s  Four  Levels  of  Evaluation.  It  is  hoped  that  this  evaluation  effort  will  help  other  online 
instructors  and  course  designers  in  their  efforts  to  design  online  courses  that  will  be  effective  and  satisfying  to  their 
students. 

Literature  Review 

The  term  “distance  education”  or  “distance  learning”  refers  to  the  teaching-learning  arrangement  in  which 
the  learner  and  teacher  are  separated  by  location  and/or  time  (Moore,  1990).  The  World  Wide  Web  (WWW)  is  a 
distributed,  hypermedia  based,  platform  independent,  architecture  for  sharing  information.  Web-based  education  is 
defined  as  education  delivered  in  whole  or  in  part  using  the  Web  and  related  technologies  (Khan,  1997).  There  are 
many  other  similar  terms  used  to  describe  web-based  education,  such  as:  online  courseware,  leamware,  distance 
education  online,  etc. 

Through  the  years,  distance  education  has  taken  advantage  of  current  technologies,  incorporating  into  the 
teaching  and  learning  environment  telecommunication  technologies  such  as  radio  and  television  broadcasting,  audio 
and  video  recording,  live,  two-way  interactive  audio,  video.  More  recently  there  has  been  a huge  growth  in  the  use 
of  synchronous  & asynchronous  computer-based  interaction  tools  on  the  Internet  or  the  World  Wide  Web  (WWW) 
(Moore  & Kearsley  1996).  Today,  Internet-based  distance  learning  is  one  of  the  most  rapidly  growing  aspects  of 
education  and  training  in  the  world. 
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While  many  argue  that  there  is  promising  future  in  the  use  of  the  Web  in  education,  there  is  no  satisfying 
answer  for  the  question  of  how  this  technology  can  best  be  used  to  approximate,  or  better,  the  traditional  on-campus 
classroom  experience.  As  stated  by  Windschitl  (1998)  and  Moore  (1990),  much  of  the  published  work  about 
distance  education  has  been  anecdotal  descriptions  of  activities  such  as  setting  up  on-line  mentoring  programs  or 
how  to  get  students  involved  in  collaborative  web-based  activities  with  other  schools.  However  there  is  not  much 
solid  research  on  the  evaluation  of  Internet-based  distance  education. 

Much  attention  was  recently  given,  including  an  article  in  the  New  York  Times , to  a study  by  Hara  and 
Kling  (2000)  that  focused  on  the  frustration  expressed  by  to  students  in  an  online  distance  education  course,  an 
extreme  level  of  frustration  that  Hara  and  Kling  refer  to  as  ‘distress.’  Certainly,  the  results  of  this  study  should 
concern  distance  educators  and  administrators  at  institutions  that  offer  distance  education  courses.  But  as  the  New 
York  Times  article  stated,  “The  report  looks  at  just  one  class  --  and  a small  one  at  that  — so  it  is  not  a survey  of 
distance  education  courses  as  a whole,  and  few  if  any  generalizations  can  be  drawn  from  it.”  Further  studies  looking 
at  the  experiences  of  online  distance  education  students  need  to  be  performed  before  any  serious  judgments  of  the 
pros  and  cons  of  this  new  educational  delivery  system  are  made.  The  present  study  will,  we  hope,  add  to  the  body 
of  knowledge  about  how  students  experience  this  new  type  of  college  course. 

Definition  of  “Distress” 

Hara  and  Kling  (2000)  defined  distance  education  ‘distress’  as  “situations  that  the  students... find 
particularly  troublesome.”  They  found  five  main  causes  of  distress  for  the  distance  students  in  the  graduate-level 
online  course  they  evaluated: 

• Social  Isolation 

• Overwhelming  Email  Communication 

• Lack  of  Instructor  Feedback 

• Technical  Problems 

• Ambiguous  Instructions 

In  this  study,  we  will  evaluate  a similar  graduate-level  course  and  see  if  the  students  report  the  same  causes  of 
distress. 

Research  Methods 

This  study  focuses  on  student  evaluations  of  an  online,  graduate-level  course  offered  by  the  School  of 
Education  at  a large  midwestem  state  university.  The  class  size  was  not  large:  1 1 students  were  registered  for  the 
course  that  semester.  The  instructor  for  the  course  under  study  was  a native  English  speaker,  and  had  previous 
experience  as  an  online  educator,  having  taught  this  course  and  one  other  course  through  online  means  in  previous 
semesters.  Students  were  located  in  Indiana,  Hawaii,  Iowa,  Tennessee,  North  Dakota,  and  Sweden.  Because  the 
online  course  would  not  have  class  meetings  in  a physical  location,  it  was  impossible  for  the  researchers  to  gather 
data  from  observation  in  class  and  interviews  with  students.  Therefore,  the  researchers  designed  web-based 
questionnaires  that  would  allow  us  to  collect  our  raw  data  via  the  Internet. 

The  researchers  utilized  a five-step  process  for  developing  these  questionnaires: 

• Interviewed  instructor 

• Reviewed  course  content/activities 

• Developed  questions  based  on  Kirkpatrick’s  Four  Levels  of  Evaluation 

• Developed  online  survey  instruments 

• Integrated  surveys  into  course  website 

In  order  to  ensure  a high  percentage  of  student  response,  the  researchers  arranged  with  the  instructor  so  that 
the  students  would  have  to  complete  the  forms  before  they  went  on  to  the  next  activity.  If  students  had  asked  not  to 
fill  out  the  form,  the  researchers  would  have  allowed  them  to  access  the  next  unit  without  submitting  the  form,  but 
none  of  the  students  pursued  this  option.  In  this  way,  the  researchers  were  able  to  get  response  to  our  forms  from  all 
the  students  that  were  actively  working  on  the  course  requirements.  Two  students  were  unable  to  complete  the 
evaluation:  one  dropped  the  course,  and  one  was  ill. 

The  questions  on  the  survey  instruments,  as  noted  above,  were  based  on  Kirkpatrick’s  Four  Levels  of 
Evaluation  (Kirkpatrick,  1998).  Each  of  the  levels  focused  on  a different  aspect  of  student  response  to  the  course. 
Our  definitions  of  the  four  levels  are  as  follows: 

• Level  1 (Student  satisfaction):  Student  satisfaction  towards  instructor,  methods  and  content 

• Course  materials 

• Course  activities 
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• Instructor  performance 

• Overall  rating  for  course 

• Level  2 (Student  learning):  Students’  gain  in  knowledge  and  skills 

• Level  3 (Transfer  of  learning):  Students’  application  of  knowledge  and  skills  to  real  job/school  environment 

• Level  4 (Cost/benefit  impact):  Analysis  of  efficiency  and  effectiveness  of  the  program  from  both  the  school’s 
and  students’  point  of  view 

It  is  important  to  note  that  for  all  of  these  levels,  we  were  only  able  to  measure  the  students’  estimation  of 
the  degree  to  which  the  course  satisfied  their  requirements  for  each  level.  In  particular,  for  Level  3 (Transfer  of 
Learning),  we  could  only  measure  the  students’  opinions  regarding  whether  or  not  they  would  be  able  to  transfer 
their  learning  to  other  settings;  we  were  not  able  to  actually  measure  whether  or  not  this  transfer  occurred,  or  would 
occur. 


Results  Of  The  Study 

In  this  section  of  the  paper,  we  present  the  results  of  the  student  course  evaluation.  The  following 
paragraphs  describe  the  most  significant  findings  of  our  study.  Overall,  the  student  responses  can  be  summarized  as 
follows,  broken  down  into  positive  and  negative  feedback; 

Positive  Student  Feedback 

Level  1 : 

Overall,  students  gave  positive  comments  regarding  this  course. 

Level  2: 

The  students  reported  a moderate  level  of  learning. 

Level  3: 

Students  expected  professional  benefits  in  the  future  from  taking  the  course. 

Level  4: 

Students  responded  that  the  course  cost  them  more  money  than  on-campus  course,  but  saved  them  time. 
The  majority  of  the  students  felt  that  the  cost/benefit  ratio  of  the  course  was  very  favorable. 

Overall,  student  reaction  to  the  course  was  positive.  In  their  evaluations,  they  provided  us  with  some 
positive  comments  about  the  course.  Most  students  were  very  confident  that  they  had  accomplished  the  learning 
objectives  of  the  course.  Students  expected  professional  benefits  in  the  future  from  taking  the  course.  The  majority 
of  the  students  commented  that  the  course  saved  them  the  time  and  effort  of  coming  to  campus  to  take  a course  in 
the  traditional  manner. 

Negative  Feedback:  ‘Distress’ 

Level  1: 

Many  students  were  not  very  satisfied  with  their  interactions  with  the  instructor. 

Level  2: 

A small  number  of  students  were  not  well  prepared  for  the  technological  requirements  of  this  course,  which 
caused  them  frustration. 

Level  4: 

Students  responded  the  course  cost  them  more  money  than  an  on-campus  course.  Also,  some  students  cited 
decreased  time  to  spend  with  their  family  and  on  their  work  as  an  opportunity  cost  of  the  course.  One  student  stated, 
“I  have  little  time  for  my  family.  (. . .)  Much  time  is  spent  with  the  mechanics  of  the  course,  which  includes  posting 
on  work  on  websites  and  submitting  it.”  Another  student  said,  “I  have  been  forced  to  reduce  some  of  my  time  at 
work.” 

Levels  1 , 2,  3: 

One  student,  in  a moment  of  “distress,”  made  the  following  statement:  "I  am  totally  frustrated.  I absolutely 
do  not  know  how  this  class  is  organized  and  how  to  access  the  information  I need.  I hate  Long  Distance  education 
and  I never  plan  to  do  this  ever  again.  It  has  made  me  rethink  even  using  the  Internet  in  my  class  at  school.  I hate 
this.  I hate  this.  I hate  this." 
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In  their  findings,  Hara  and  Kling  noted  two  primary  sources  of  students’  ‘distress’:  technological  problems, 
and  the  instructor’s  online  communication  practices.  Our  study  found  the  same  issues  were  the  primary  concerns  of 
the  students  in  this  course. 

The  course  required  students  to  navigate  through  various  websites,  send  and  respond  to  email  messages, 
and  use  a web-based  conferencing  tool  for  class  discussion  purposes.  Some  students  reported  difficulties  with 
utilizing  the  technologies  required  for  these  tasks.  A few  students  complained  about  the  way  the  course  website, 
which  was  developed  by  the  instructor,  was  structured. 

In  this  course,  many  students  were  not  very  satisfied  with  their  online  interactions  with  the  instructor. 
While  the  students  recognized  the  instructor’s  knowledge  of  the  subject  matter,  the  majority  of  the  students  (5  of  9 
respondents)  did  not  give  positive  evaluations  of  their  online  interactions  with  the  instructor. 

In  comparing  our  findings  with  those  of  Hara  and  Kling,  our  students  reported  that  they  shared  3 of  5 
causes  of  ‘distress’  with  the  students  in  the  Hara  and  Kling  study. 

• Social  Isolation  (not  found) 

• Overwhelming  Email  Communication  (not  found) 

• Lack  of  Instructor  Feedback  (found) 

• Technical  Problems  (found) 

• Ambiguous  Instructions  (found) 

Thus,  the  majority  of  the  causes  of ‘distress’  were  the  same  for  both  courses. 

Recommendations 

After  a careful  review  of  the  student  responses  to  the  course  evaluation  instruments,  and  our  analysis  of  the 
causes  of  distance  education  student  ‘distress,’  we  have  developed  the  following  recommendations  for  online  course 
instructors. 

• The  instructor  should  review  her  practices  in  responding  to  students’  email  and  web-based  conferencing  posts, 
to  ensure  that  she  is  providing  sufficient  and  appropriate  feedback. 

• The  instructor  should  specify  the  technological  requirements  in  the  syllabus,  and  arrange  for  technical  support. 

• The  instructor  should  conduct  a usability  tests  on  the  course  requirements  and  other  instructions  to  ensure  that 
they  are  clear  and  non-ambiguous. 

• Even  though  the  students  did  not  report  that  social  isolation  was  a major  factor  causing  ‘distress’,  we 
recommend  that,  due  to  its  prevalence  in  the  Hara  and  Kling  study,  the  distance  education  program  should 
provide  an  on-campus  face-to-face  orientation  for  its  distance  students.  For  students  that  cannot  be  on-campus, 
a videotape  should  be  provided. 

Limitations 

The  research  design  for  this  study  faced  a number  of  limitations,  chief  among  them  being  a limited  time 
frame  for  administering  the  evaluation.  Other  limitations  were: 

• Small  sample  size  (though  entire  class) 

• Education  students  may  be  atypical 

• Lack  of  follow-up  evaluation 

• Lack  of  ability  to  measure  Level  3 (Transfer  of  Learning) 

Conclusions 

Of  course,  as  with  Hara  and  Kling’s  study,  the  generalizability  of  the  data  in  this  survey  is  necessarily 
limited  because  it  is  only  based  on  the  student  evaluations  of  one  course.  But  as  more  online  courses  are  delivered, 
and  more  students  have  the  opportunity  to  voice  their  opinions  on  the  pedagogy  and  technology  of  these  courses,  the 
researchers  feel  that  it  is  important  that  their  responses  are  shared  with  distance  education  instructors  and 
administrators  of  online  programs. 

Based  on  the  experiences  of  the  students  in  this  online  course,  the  researchers  have  two  primary 
recommendations  for  others  developing  such  courses. 

First,  to  ensure  that  the  students  feel  “The  Pains  of  Innovation”  as  Hara  and  Kling  refer  to  it,  to  the  least 
degree  possible,  the  instructor  should  clearly  specify  the  technological  requirements  at  the  outset  of  the  course  and 
should  supply,  or  arrange  for  others  to  supply,  the  necessary  technical  support. 

Second,  more  attention  also  needs  to  be  paid  to  developing  and  refining  methods  of  moderating  and 
monitoring  online  discussions  with  students.  The  researchers  suggest  that  instructors  should  review  their  practices 
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regarding  responding  to  students’  email  and  web-based  conferencing  posts,  so  ensure  that  they  are  providing 
sufficient,  appropriate,  and  reasonably  prompt  feedback. 
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Abstract 

A win-win  partnership  has  been  created  in  the  University  of  Hawaii  Educational  Technology  Master’s 
program  allowing  graduate  students  to  tiy  out  their  new  skills  by  teaching  faculty  to  integrate  technology . These 
efforts  have  been  in  concert  with  USDOE  Preparing  Tomorrow’s  Teachers  for  Technology  [PT3]  and  Eisenhower 
grant  projects  designed  to  provide  professional  development  for  a new  designation  of  “technology  intensive” 
courses , Workshops  and  individual  assistance  are  provided  by  graduate  students  in  practicum  and  internship 
courses . 

Students  in  instructional  design  and  technology  programs  learn  valuable  skills  that  can  be  transferred  to  the 
real  world  of  work.  However,  this  transfer  may  be  less  than  successful  if  appropriate  guided  opportunities  are  not 
provided  to  bridge  the  gap  from  school  to  work.  win-win  situation  has  been  created  in  the  University  of  Hawaii’s 
Educational  Technology  (ETEC)  master’s  progra/n  that  allows  graduate  students  to  try  out  their  new  skills  by 
teaching  College  of  Education  (COE)  faculty  to  integrate  technology  into  their  courses. 

After  completing  core  courses  in  educational  technology  including  one  year  of  instructional  design, 
students  are  required  to  take  an  Educational  Technology  Practicum.  The  practicum  provides  students  a guided 
setting  for  trying  out  their  new  skills  in  a real-world  experience.  This  model  may  be  typical  of  many  programs,  but 
what  is  unusual  is  the  partnership  that  has  been  formed  by  graduate  students  and  faculty  to  provide  a valuable 
service  for  the  College  of  Education*.  In  this  course,  students  work  as  a group  to  design  technology  workshops. 

These  efforts  have  been  in  concert  with  the  US  Department  of  Education’s  “Preparing  Tomorrow’s 
Teachers  to  Use  Technology"  and  Eisenhower  Professional  Development  Fund  grant  projects  in  the  Educational 
Technology  department,  These  projects  have  been  designed  to  provide  professional  development  for  a new 
designation  of  “technology  intensive’’  (TI)  courses. 

Each  year  the  graduate  students  in  the  Educational  Technology  Practicum  have  unanimously 
chosen  to  continue  conducting  technology  workshops  to  hone  their  instructional  design  skills  and  provide  the  service 
to  the  COE  faculty.  To  maximize  involvement,  the  practicum  course  enrollment  is  purposefully  kept  small,  ranging 
from  five  to  ten  students. 

Students  in  the  course  are  ETEC  majors  who  have  completed  their  first  year.  Required  in  the  first  year  are 
courses  in  which  students  use  a variety  of  media  to  analyze  systems,  conduct  needs  assessments,  explore  change 
theories,  create  instructional  hierarchies  and  write  formal  instruction.  Practicum  students  come  from  a variety  of 
backgrounds.  As  most  Educational  Technology  courses  are  taught  in  the  evenings,  many  students  have  full  time 
jobs  or  are  practicing  teachers.  The  varying  experiences  they  have  outside  of  the  department  bring  numerous 
viewpoints  and  skills  to  the  group. 

Students  are  allowed  as  much  leeway  as  possible  in  determining  and  carrying  out  their  design  plan.  The 
only  requirements  are  that  they  follow  standard  design  procedures.  Educational  Technology  students  in  the  first 
year  use  Dick  and  Carey’s  (1996)  systems  approach  for  designing  instruction.  The  practicum  gives  students  the 
opportunity  to  directly  apply  this  approach  to  the  real  experience  of  designing  and  conducting  workshops. 

The  first  step  of  the  Dick  and  Carey  model  is  assessing  need  and  identifying  goals.  In  order  for  the 
practicum  to  be  win-win,  students  identify  their  own  needs  and  goals  as  well  as  those  of  the  faculty.  Initial  class 
discussions  are  ones  in  which  students  identify  their  WIIFMs  or  “what’s  in  it  for  me?”  They  discuss  what  they 
would  like  to  learn  and  ultimately  like  to  get  out  of  the  course.  Students  also  identify  skills  they  have  in  the  group 
especially  with  regard  to  technology  knowledge  that  can  be  shared  with  the  faculty. 

The  win-win  situation  is  fostered  by  the  students  considering  their  own  needs  concurrently  with  those  of 
the  faculty.  Needs  analysis  studies  of  the  faculty  have  been  conducted  frequently  in  the  College  of  Education  to  find 
out  how  technology  is  being  used,  identify  the  training  needs  of  faculty,  and  provide  recommendations  for  the  future 
growth  of  technology  applications  in  the  COE.  The  most  recent  study  was  conducted  in  1999  by  Ho,  Sherry, 
Speitel,  & Walton. 
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Responses  from  the  faculty  regarding  what  technologies  they  feel  are  relevant  and  interesting  become  a 
valuable  guide  for  the  students.  Students  now  match  what  they  would  like  to  teach  and  learn  with  what  the  faculty 
want  to  learn.  From  this  teaming  emerge  the  student  goals  for  themselves,  the  practicum  and  for  teaching  the 
faculty. 

Instrumental  in  this  planning  phase  are  Dick  and  Carey’s  second  and  third  steps,  analyzing  content  and 
audience.  Students  must  consider  the  faculty  and  themselves  as  the  audience  in  order  for  the  situation  to  remain  win- 
win.  This  is  when  they  begin  to  reconcile  what  the  faculty  want  to  gain  and  what  they  themselves  want  to  gain  from 
this  experience.  Resources  are  an  important  factor  when  determining  content.  Questions  asked  at  this  point  are: 
What  equipment  do  we  have  available  to  us  in  the  department?  What  equipment  do  faculty  have  available  to  them 
in  their  departments?  Who  within  the  group  has  the  skills  to  teach  what  the  faculty  want  to  learn?  Where  can  we 
locate  resources  and  expertise  outside  of  our  group?  With  these  questions  answered,  a series  of  topics  for  the 
workshops  are  conceived  and  scheduled  across  the  semester.  An  average  of  5 workshops  are  scheduled  over  each 
practicum  semester  giving  students  initial  planning  time  and  a week  between  each  workshop  event. 

Workshop  topics  covered  in  the  Fall  2000  semester  include:  Online  collaboration  supported  by  WebCT, 
Electronic  portfolios,  Inspiration,  the  mind  mapping  software,  Dreamweaver  for  web  authoring,  Adobe  Acrobat,  and 
preventative  computer  maintenance,  among  others.  The  Fall  2000  group  also  organized  an  open  house  event  for  the 
COE’s  Technology  Learning  Center.  Past  groups  have  conducted  mini-conferences  featuring  break-out  sessions 
with  faculty  members  who  technology  intensive  courses. 

Students  must  now  recruit  for  and  advertise  the  workshops.  Email  messages,  fliers  in  faculty  boxes  and 
word  of  mouth  are  the  most  common  methods  of  advertising.  Registration  rates  provide  the  students  with  informal 
data  on  what  the  faculty  are  truly  interested  in.  Barring  scheduling  conflicts  and  other  commitments,  students 
assume  that  workshops  with  high  registration  rates  are  covering  topics  more  relevant  and  interesting  to  faculty  than 
those  which  are  not  being  registered  for.  Well  designed  registration  forms  will  also  collect  additional  information 
about  participants’  skill  levels,  position  and  interests. 

The  fourth  step  of  the  Dick  and  Carey  model,  writing  objectives,  is  now  completed.  The  COE  needs 
assessments  in  which  faculty  have  rated  their  confidence  and  skill  levels  with  specific  technologies  prove  again  to  be 
a valuable  resource.  Based  on  the  skill  levels  reported  by  faculty  and  the  topics  that  have  been  decided  upon, 
objectives  are  written  for  each  workshop.  Students  must  also  take  into  account  how  much  content  can  reasonably  be 
covered  in  the  two-hour  workshop  time  period.  An  example  of  an  objective  from  a Microsoft  Word  workshop  is 
“participants  will  be  able  to  save  a Microsoft  Word  text  file  in  HTML  format.”  A Photoshop  workshop  example  is 
“participants  will  be  able  to  import  layers  from  other  files.” 

Students  now  take  Dick  and  Carey’s  fifth  step  and  develop  their  test  or  evaluation  tools.  Evaluations  are 
directly  aligned  with  the  objectives  for  each  workshop  and  are  administered  at  the  end  of  each  workshop.  Surveys 
are  the  most  common  instrument  used  and  look  primarily  for  changes  in  the  affective  domain.  Faculty  report  on 
their  confidence  levels  with  technology  before  and  after  the  workshop,  rank  the  personal  relevance  of  the 
information  presented  and  comment  on  their  interest  in  the  topics  and  guest  speakers.  Questions  are  directly  aligned 
to  the  content  presented  in  each  workshop.  It  is  important  to  note  that  students  are  encouraged  to  make  the 
evaluation  instruments  non-threatening  to  faculty  and  to  avoid  making  faculty  members  feel  they  are  being  tested. 
Students  are  also  encouraged  not  to  solicit  information  which  may  be  perceived  as  judging  the  quest  speakers. 
Appropriate  space  is  allotted  for  suggestions  and  comments  from  the  faculty  on  how  to  improve  and  revise  further 
workshops. 

Dick  and  Carey’s  model  now  calls  upon  the  students  to  develop  a strategy.  Students  are  reminded  to 
review  Smith  and  Ragan’s  (1993)  organizational,  delivery  and  management  components  of  instructional  strategies. 
Organizational  aspects  encompass  the  scope  and  extent  of  content.  Content  is  made  specific  and  thought  given  to 
how  thoroughly  it  can  be  covered  in  the  workshop  time  period.  Delivery  components  include  issues  of  appropriate 
media  selection  and  workshop  style.  Appropriate  media  compliment  and  enhance  workshop  topics.  Depending  on 
content,  one  or  more  of  the  following  delivery  styles  is  chosen:  student  presentations,  guest  speakers,  hands-on 
activities,  discussions,  and  use  of  examples  or  non-examples.  Student  presentation  of  content  most  often  involves 
multimedia  presentation  software  such  as  PowerPoint.  Content  is  usually  demonstrated  step  by  step  while  the 
faculty  observe  or  follow  along  with  paper-based  materials. 

Guest  speakers  are  a popular  way  for  students  to  provide  faculty  with  examples  of  real  classroom 
applications  for  the  technologies  presented.  Guest  speakers  are  chosen  for  their  motivational  style  and  positive 
attitudes  as  well  as  their  personal  and  innovative  methods  of  technology  integration.  The  goal  of  having  a guest 
speaker  is  to  motivate  and  stimulate  the  faculty  to  generate  ideas  about  technology  integration  in  their  own 
classrooms. 


136 


145 


A common  workshop  agenda  begins  with  a guest  speaker  followed  by  student  presentation  of  content  using 
multimedia  presentation  software.  This  portion  of  the  workshop  tends  occur  in  a classroom-style  setting.  Following 
a short  break,  the  workshop  moves  to  the  teaching  lab  where  participants  can  work  individually  or  in  pairs  on 
computers  to  practice  the  technology  applications.  This  type  of  workshop  has  proven  to  be  a successful  blend  of  the 
technology  integration  issues  the  ETEC  department  students  and  faculty  want  to  teach  and  the  hardware  and 
software  specifics  the  COE  faculty  want  to  learn. 

Hands-on  activities  are  popular  for  most  workshops.  The  COE  has  two  teaching  labs,  one  Macintosh  and 
one  IBM  compatible  in  which  hands-on  activities  can  be  conducted.  Each  room  has  approximately  20  computers. 
Hands-on  activities  provide  participants  with  the  opportunity  to  practice  their  new  technology  skills  while  the 
practicum  students  act  as  helpers.  When  faculty  have  been  provided  opportunities  to  attend  workshops  solely 
regarding  integration,  response  has  been  lower  than  when  integration  issues  are  paired  with  hands-on  activities  in 
which  they  learn  hardware  and  software  specifics.  Participants  have  reported  that  hands-on  activities  increase  their 
confidence  levels  with  technology  and  are  more  exciting  than  presentations  alone.  The  practicum  situation,  with  5 
to  10  students  available,  presents  the  opportunity  to  provide  one-on-one  assistance  during  the  instruction. 

Discussions  are  considered  for  a workshop  when  content  may  relate  to  but  not  be  specifically  based  on 
technology  skills.  For  example,  a discussion  about  technology  standards  and  their  application  in  the  classroom 
might  be  more  appropriate  than  a hands-on  activity  about  the  same  topic. 

Students  may  also  employ  the  use  of  examples  and  non-examples  to  demonstrate  effective  applications  of 
the  content  they  present.  While  guest  speakers  provide  examples  of  a positive  technology  applications,  students 
may  also  consider  the  use  of  non-examples  or  negative  applications  of  technology.  Non-examples  present  a 
negative  usage  of  a technology  with  the  intent  of  reinforcing  the  validity  of  positive  usages. 

In  conjunction  with  determining  what  types  of  media,  materials,  speakers,  and  format  should  be  used, 
students  must  also  address  the  management  and  practicalities  of  facilitating  the  events.  Good  management  strategies 
become  imperative  at  this  time.  Tasks  include  but  are  in  no  way  limited  to  locating  and  reserving  facilities  and 
equipment  and  soliciting  and  confirming  guest  speakers.  Students  now  decide  not  only  how  the  workshops  will  be 
managed  but  how  they  will  manage  themselves  as  a group  to  achieve  the  goals  they  have  set  for  themselves.  They 
create  extensive  task  lists  and  checklists  then  assign  roles  to  members  of  the  group.  Different  groups  organize 
themselves  differently.  Some  groups  have  found  it  beneficial  for  all  group  members  to  work  on  and  contribute  to 
each  workshop  while  some  groups  assign  individuals  as  leaders  of  particular  workshops  with  the  rest  of  the  group 
acting  as  support  to  that  leader.  The  assignments  of  these  roles  tend  to  have  long  term  impact  on  the  success  of  the 
practicum.  Although  students  are  encouraged  to  and  do  revise  throughout  the  course  these  initially  assigned  roles 
tend  to  remain  consistent  throughout  the  semester.  The  most  success  has  occurred  when  roles  are  assigned  which 
align  with  an  individual’s  strengths  as  well  as  their  WIIFMs.  For  example,  a student  that  would  like  management 
experience  often  becomes  an  overall  leader  of  the  group.  A student  with  extreme  stage  fright  and  a desire  for  better 
presentation  skills  may  not  be  the  most  appropriate  person  to  be  the  main  presenter  for  workshops.  On  the  other 
hand,  the  instructor  may  encourage  this  person  to  be  a presenter  at  least  once  to  provide  a learning  experience  and 
opportunity  to  develop  those  skills. 

Following  the  progression  of  the  Dick  and  Carey  model  students  then  develop  materials  for  the  workshops. 
Bearing  in  mind  the  delivery  style  and  content  of  the  workshops,  accompanying  materials  are  created  ranging  from 
web  to  paper-based  instructional  booklets.  Multimedia  presentations  are  most  often  used  to  present  content  and 
demonstrate  software  applications.  The  design  of  materials  incorporates  a variety  of  instructional  styles.  Some 
workshops  may  use  instructional  materials  during  hands-on  activities  while  participants  follow  along  step  by  step. 
Other  materials  are  designed  to  be  used  after  the  workshop.  An  example  of  an  after- works  hop  resource  is 
instruction  on  advanced  features  of  a technology  which  were  not  addressed  in  the  workshop.  Other  after-workshop 
resources  have  included  workshop  series  web  sites  with  general  information,  workshop  schedules  and  PDF  files  of 
workshop  materials.  Fall  1999  semester’s  web  address  is  http://www2.hawaii.edu/~ganne/PauHana.  Students  are 
encouraged  to  be  “resource  linkers”  when  developing  materials,  locating  available  existing  resources  and  materials 
and  incorporating  them  into  their  own. 

The  production  of  materials  brings  up  issues  of  theme  and  appearance.  This  issue  most  often  reflects  the 
management  style  of  the  group.  Groups  who  are  extremely  collaborative  with  all  group  members  working  on  all 
workshops  tend  to  devise  on  an  overall  theme  and  look  for  the  workshop  series  and  accompanying  materials  while 
groups  in  which  leaders  emerge  for  specific  workshops  may  develop  different  looks  for  different  workshops. 
Accompanying  materials  have  included  advertising  fliers,  registration  forms,  evaluation  surveys,  instructional 
booklets,  web  sites,  agendas  and  certificates  of  acknowledgement  given  to  guest  speakers. 

Dick  and  Carey’s  model  stresses  that  formative  evaluation  and  revisions  take  place  throughout  the  design 
process.  When  the  practicum  workshops  have  been  developed,  formative  evaluation  is  conducted  through 
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rehearsals.  A “dry-run”  of  each  workshop  is  one  of  the  few  requirements  of  the  course.  The  opportunity  to  go 
through  all  materials  and  instruction  in  real  time  and  in  the  actual  setting  provides  students  with  practice  that 
improves  their  presentations  and  workshops  dramatically.  The  group  and  the  instructor  observe  and  participate  in 
the  workshop  rehearsals.  Following  rehearsal,  presenters  critique  themselves  on  aspects  they  felt  were  problematic 
and  the  rest  of  the  group  is  encouraged  to  provide  additional  opinions.  They  catch  many  mistakes  and  timing 
misjudgments.  Lastly,  the  instructor  provides  feedback  and  suggestions.  Rehearsal  sessions  take  place  one  week 
before  workshop  events  and  students  make  final  revisions  throughout  the  week. 

The  course  period  is  2.5  hours  long  and  workshop  sessions  are  2 hours.  The  extra  half  hour  is  used  for  set- 
up prior  to  and  breakdown  after  the  event.  Details  overseen  by  students  for  workshop  events  include  registration 
tables,  pre-printed  nametags  for  participants,  sign  in  sheets  and  refreshments.  Students  dress  more  formally  for 
workshops  than  they  do  for  class.  Workshop  participants  are  asked  to  complete  evaluation  forms  at  the  end  of  each 
workshop.  Students  provide  “prizes”  such  as  candy  for  those  who  complete  evaluation  forms  to  raise  the  turn-in 
rate.  Following  each  workshop  the  instructor  and  the  practicum  group  conduct  another  formative  evaluation  session, 
or  “de-briefing”.  They  discuss  what  went  right  and  what  went  wrong.  Participant  evaluations  are  reviewed  and 
discussed.  Students  reflect  on  the  process  of  planning  and  conducting  the  workshop  and  suggestions  are  offered  as 
to  what  will  make  the  next  workshops  better. 

The  last  stage  of  the  practicum  course  and  the  Dick  and  Carey  model  concentrates  on  summative 
evaluation.  Students  are  required  to  produce  a final  report  about  the  course.  In  it  they  include  the  objectives  for 
each  workshop,  their  WIIFMs  or  “what’s  in  it  for  me”  statements,  what  went  wrong  and  what  went  right  in  each 
workshop,  their  reflections  and  suggestions  for  future  workshops  and  all  the  original  materials  they  created  in  the 
course.  This  document  is  addressed  to  the  next  practicum  group  as  advice  from  those  who  just  went  through  the 
process.  Examples  of  what  went  right  and  wrong  statements  for  an  Avid  cinema  workshop  include:  “I’m  glad  I 
created  a personal  timeline  to  help  me  get  all  my  tasks  done,  especially  for  the  day  before  the  event.  I’m  also  glad  I 
used  templates  in  the  workshop  that  faculty  could  use.  I should  have  tested  the  hands-on  part  on  the  workshop  day. 
I should  have  practiced  more  to  make  my  timing  smooth”.  Reflections  about  the  course  include  statements  such  as: 
“Listening  to  team  feedback  is  important,  I learned  to  ask  for  help  from  the  team,  and  I learned  the  importance  of 
preplanning”.  Examples  of  suggestions  for  the  future  include:  revise,  revise,  revise,  plan  ahead,  delegate  small  tasks 
out  to  the  whole  team,  and  rehearse,  rehearse,  rehearse.  Some  practicum  groups  have  created  fliers  with  “10  tips”  for 
the  next  group  offering  advice  and  suggestions  of  how  to  get  started  and  what  to  avoid  in  the  planning  and  execution 
phases  of  the  workshops. 

Over  the  four  years  the  course  has  conducted  faculty  workshops,  student  evaluations  have  been  consistently 
high  and  the  course  is  always  full.  Faculty  in  the  department  who  employ  students  who  have  been  through  the 
course  report  that  these  students  tend  to  have  excellent  presentation  skills  and  high  levels  of  confidence.  Workshop 
sessions  are  consitently  well  attended  and  evaluations  positive.  In  this  way  the  process  is  win-win.  Students  get  a 
real  world  design  and  presentation  experience  and  the  faculty  get  free,  in-house  technology  workshops  tailored 
specifically  to  their  needs. 

Following  the  practicum  workshops,  faculty  are  recruited  to  be  involved  with  grant  projects  and  redesign 
their  courses  to  be  technology  intensive.  The  grants  provide  student  technology  consultants  who  meet  weekly  with 
the  faculty  members  for  one-on-one  mentoring  to  help  them  achieve  this  goal.  The  National  Council  on 
Accreditation  of  Teacher  Education  (1999)  suggests  that  mentoring  and  providing  feedback  is  an  effective  method 
of  professional  development  in  technology  integration. 

The  student  consultants,  who  are  primarily  students  in  the  Educational  Technology  masters  program,  again, 
have  an  opportunity  to  use  instructional  design  skills  in  their  approach  to  faculty  mentoring.  As  consultants  they:  (a) 
conduct  individual  needs  assessments  of  faculty  members,  (b)  set  goals  for  the  professional  development;  (c) 
provide  expertise  in  creating  a revised  curriculum;  (d)  assist  in  improving  technical  skills  to  help  faulty  members 
reach  their  technology  integration  goals;  and  (e)  provide  encouragement  and  guidance  in  course  design  and 
technology  integration.  The  goal  and  final  product  of  the  mentoring  is  a technology  intensive  course  proposal  in 
which  the  faculty  member  outlines  how  they  have  restructured  one  or  more  of  their  courses  using  the  technology 
intensive  standards.  These  standards  directly  parallel  the  International  Society  for  Technology  in  Education  (ISTE) 
standards  and  were  developed  through  a grant  project  at  the  University  of  Hawaii  in  1996.  The  essence  of  a TI 
course  is  that  both  the  instructor  and  student  are  actively  using  technology  in  the  teaching  and  learning  process.  An 
example  of  a redesigned  TI  course  would  be  one  in  which  students  and  instructor  are  communicating  via  email,  the 
instructor  is  using  multimedia  software  to  present  content  in  class  and  students  are  giving  multimedia  presentations 
in  place  of  papers.  Redesigned  courses  also  often  include  the  inclusion  of  web  based  activities  such  as  electronic 
bulletin  boards,  chat  rooms,  threaded  discussions  and  web  site  construction,  or  digital  video  and  electronic 
portfolios. 
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Based  on  grant  project  evaluations  and  course  evaluations,  students  and  faculty  have  greatly  increased  their 
skill  levels  and  their  confidence  with  technology  and  its  integration  in  the  classroom.  This  partnership  has  created  a 
synergistic  effect  that  is  best  described  by  Covey  (1994).  “Win-win  means  that  agreements  or  solutions  are 
mutually  beneficial,  mutually  satisfying.  With  a win-win  solution,  all  parties  feel  good  about  the  decision  and  feel 
committed  to  the  action  plan.”(p.l  17)  The  evaluations  by  both  faculty  and  students  have  confirmed  that  their 
partnership  meets  this  ultimate  goal. 
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Abstract 

In  this  study,  we  examined  differences  between  two  populations  of  composition  students  over  the  course  of 
a semester  in  their  perceptions  of  collaboration  as  reported  in  pre-  and  post-surveys.  In  addition,  using  an 
assessment  rubric  we  developed,  we  examined  students'  audience  awareness  as  demonstrated  in  their  writing , Both 
groups  of  students  were  taught  by  the  same  instructor  and  had  the  same  writing  assignments;  both  classes 
incorporated  a pedagogy  of  collaborative  learning  to  help  students  develop  a sense  of  belonging  to  a discourse 
community;  and  both  sections  used  networked  computers  as  learning,  writing,  and  communication  tools.  The 
distinguishing  variable  between  the  two  sections  was  the  absence  of  face-to-face  communication  among  students  in 
the  online  class.  In  our  study,  we  posed  the  following  research  questions:  In  a comparison  of  students  in  a on- 
campus  networked  writing  class  with  students  in  an  off-campus  online  writing  class,  (a)  Do  students  differ  in  how 
they  value  collaboration  during  the  writing  process?  (b)  Do  students  differ  in  how  they  address  audience  needs  in 
their  writing  products ? (c)  Do  students  differ  in  their  level  of  satisfaction  with  their  learning  experience ? 
Conclusions  from  this  study  are  the  following:  (!)  Online  students  tend  to  be  more  independent  learners,  valuing 
collaboration  less  than  do  on-campus  students;  (2)  Online  interaction  appears  to  increase  audience  awareness  in 
students ' writing.  (3)  Students  in  both  sections  reported  positive  experiences  with  their  respective  classes. 

The  field  of  composition  has  undergone  two  significant  paradigm  shifts  in  the  last  twenty  years:  from  a 
focus  on  the  product  of  writing  to  a focus  on  the  process,  and  from  a focus  on  the  writer  as  a solitary  individual  to  a 
focus  on  the  writer  as  part  of  a discourse  community,  “a  group  of  people  with  similar  goals  and  interests  who 
constitute  themselves  with  a characteristic  language”  (Bruffee,  1993,  p.  223).  A more  recent  development,  the  use  of 
computer-mediated  instruction  in  the  college  composition  classroom,  has  facilitated  both  paradigm  shifts. 

At  the  University  of  Colorado  at  Colorado  Springs  (UCCS),  for  example,  all  writing  classes  are  taught  in 
networked-computer  classrooms,  where  students  interact  both  through  face-to-face  and  through  computer-mediated 
communication.  In  this  environment,  students  have  increased  opportunities  to  engage  in  the  writing  process  through 
collaborative  partnerships  with  other  students.  The  networked  computers  become  conduits  for  linking  students  with 
one  another,  thereby  extending  the  discourse  community  in  the  classroom  through  a second  framework  via 
cyberspace:  shared  virtual  “chambers”  where  continued  interaction  takes  place  and  where  a repository  of  student 
work,  both  individually  and  collaboratively  produced,  resides  as  a catalyst  for  further  dialectic. 

At  the  same  time,  universities  are  experimenting  with  the  delivery  of  writing  courses  exclusively  via 
distance  learning,  without  face-to-face  interaction.  In  the  fall  1999  semester,  UCCS  joined  the  ranks  of  these 
universities  by  delivering  one  section  of  freshman  composition  as  an  online  course. 

Computer-Mediated  Communication 

Computer-mediated  communication  (CMC)  changes  both  the  quality  and  quantity  of  communication  by 
allowing  time  for  critical  reflection  and  greater  involvement  in  discussion  than  is  allowed  in  the  traditional 
classroom,  where  one  or  two  students  may  monopolize  the  conversation  (Berge,  1995;  Fishman,  1997;  Wells,  1992). 
Group  conferencing  appears  to  decrease  the  emergence  of  a group  leader,  allowing  more  students  a greater  role 
(Harasim,  1990;  Warschauer,  1997).  Student-directed  conversation  and  participation  level  is  higher  in  the  CMC 
classroom,  which  shifts  the  role  of  the  teacher  from  content  expert  to  facilitative  guide  (Wells,  1992).  CMC  thus 
enhances  peer-to-peer  discussion  (Jonassen  et  al.,  1995),  and  participation  is  fairly  evenly  distributed  among 
students.  Students  report  that  they  work  harder  and  produce  higher  quality  work,  since  work  is  visible  to  their  peers 
(Oblinger  & Maruyama,  1996).  In  addition,  computer  conferencing  tools  foster  critical  thinking  and  active  learning 
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by  providing  an  electronic  space  for  reflective  journal  writing,  critical  analysis,  and  peer  and  instructor  facilitation 
(Bonk  & Reynolds,  1997).  Instructors  are  able  to  observe  students’  contributions  to  discussion,  obtain  a record  of 
the  discussion  for  future  feedback,  participate  in  the  discussion  to  model  critical-thinking  skills,  and  ask  questions  to 
coach  critical  thinking,  providing  expertise  when  necessary  (Wagner  & McCombs,  1995). 

Collaboration 

Collaboration  enhances  connectivity  and  socio-emotional  commitment  to  the  learning  process  by  involving 
students  as  active  participants  in  the  learning  process  (Sharan,  1980;  Oliver  & Reeves,  1994).  Students  achieve 
greater  cognitive  development  working  together  than  they  do  working  individually  (Sharan;  Oliver  & Reeves). 
Collaboration  can  contribute  to  the  active  construction  of  meaning,  through  idea  generating  (divergent  thinking), 
idea  linking  (convergent  thinking),  and  idea  structuring  (categorization  and  classification)  (Harasim,  1990).  Online 
collaboration,  with  its  emphasis  on  both  reflection  and  interaction,  can  amplify  the  process  of  knowledge 
construction  (Warschauer,  1997).  Collaborative  problem  solving,  involving  both  conversation  and  issue-based 
discussion,  supports  intentional  learning  and  develops  critical-thinking  skills  (Duffy,  Dueber,  & Hawley,  1998). 
Web-based  tools  such  as  e-mail,  electronic  partnerships,  project-based  learning,  and  synchronous  or  asynchronous 
conferencing  foster  collaborative  learning  (Bonk  & Reynolds,  1997). 

Advantages  and  Limitations  of  Asynchronous  Collaboration 

While  the  effectiveness  of  these  collaborative  tools  has  not  been  extensively  studied,  asynchronous 
communication  and  online  collaboration  have  some  recognizable  strengths  and  weaknesses.  Strengths  of  computer- 
mediated  collaboration  include  student  enthusiasm,  more  time  on  task,  and  student  satisfaction  (Shotsberger,  1996; 
Kemer,  Penner-Hahn,  Berger,  & Dershimer,  1997).  Students  appear  to  like  CMC,  find  the  instructor  is  more 
accessible,  and  find  problem-based  learning  and  case-study  learning  more  useful  than  they  are  in  a traditional 
classroom.  Disadvantages  include  communication  anxiety,  feelings  of  disconnectedness  from  conversational  thread, 
and  frustration  over  delayed  feedback.  Additionally,  making  decisions  from  group  consensus  can  be  time-consuming 
(Harasim,  1990;  Warschauer,  1997),  while  software  and  hardware  problems  may  limit  interaction  (Oliver  & Reeves, 
1994).  One  of  the  biggest  drawbacks  to  asynchronous  collaboration  is  the  lack  of  visual  and  verbal  cues  provided  by 
face-to-face  interaction.  (Lehman,  1995). 

Online  Students  vs.  Traditional  Students 

A comparison  of  online  and  traditional  students  shows  that  online  students  learn  on  average  as  well  as 
traditional  students,  with  respect  to  midterms,  finals  and  grades.  More  mature  and  better  students  learn  more,  while 
students  who  lack  good  study  habits  and  have  difficulty  writing  and  reading  learn  less  (Harasim,  1990).  Self- 
discipline  is  a crucial  element  of  success  in  online  learning.  Hiltz  (1990)  examined  learning  in  online  and  traditional 
classes  using  pre-/post-questionnaires,  case  studies,  institutional  data,  interviews  with  students  and  faculty,  and 
survey  of  dropouts.  She  found  no  significant  difference  between  mastery  in  the  online  class  and  traditional 
classrooms.  In  fact,  the  grades  for  students  in  the  online  computer  science  class  were  better  than  the  grades  for 
traditional  students.  She  concludes  that  online  students  learn  as  well  as  traditional  students.  Simonson,  Schlosser, 
and  Anderson  (1994)  concur:  Students  who  are  motivated,  prepared,  and  intelligent  can  potentially  learn  as  much 
online  as  in  a traditional  classroom.  While  attrition  rates  are  generally  higher  for  online  students,  researchers  have 
found  little  correlation  between  performance  outcomes  and  individual  characteristics,  especially  for  mature  learners 
(Kember,  1990). 

Collaborative  Learning,  Writing,  and  Audience  Awareness 

Collaborative  learning  in  higher  education  has  been  underused  and  frequently  misunderstood.  Bruffee 
(1993)  posits  that  knowledge  is  a social  construct,  that  learning  is  a social  process,  and  that  writing  is  central — not 
ancillary — to  collaborative  learning  and  the  construction  of  knowledge.  In  addition,  Oakeshott  (1962),  Sergiovanni, 
(1996),  Latour  and  Woolgar  (1986)  all  emphasize  the  importance  of  social  dialectic  in  the  construction  of 
knowledge.  The  use  of  technology  in  the  teaching  of  writing  has  also  been  widely  discussed  (Hawisher,  LeBlanc, 
Moran,  & Selfe,  1996).  However,  although  researchers  have  documented  a variety  of  benefits  from  the  integration 
of  technology  in  the  composition  class  (Carbone,  1993;  Klem  & Moran,  1992;  Mason,  Duin,  & Lammers,  1994),  the 
data  about  writing  improvement  are  less  clear. 

Current  approaches  to  audience  include  historical  studies  (Willard  & Brown,  1990;  Willey,  1990),  studies 
of  writers’  audience  awareness  during  the  writing  process  (Moffett,  1968),  studies  of  audience  as  a discourse 
community  (Enos,  1990;  Rafoth,  1990;  Roth,  1990;  Mangelsdorf,  Roen,  & Taylor,  1990),  and  links  between' 
audience  awareness  and  syntactic  and  lexical  features  (Rubin  & O'Looney,  1990).  This  interest  in  audience  is  related 
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to  the  increased  focus  on  examining  composition  from  a social  constructivist  perspective  (Bruffee,  1986).  In 
addition,  an  ongoing  debate  centers  on  whether  writers  “invoke”  a fictionalized  audience  or  “address”  an  actual 
audience.  Scholars  agree,  however,  that  actual  readers  can  have  a powerful  effect  on  writers  (Long,  1990;  Ede  & 
Lunsford,  1984;  Porter,  1992). 

Research  Design  and  Methodology 

In  this  study,  we  examined  differences  between  two  populations  of  composition  students  over  the  course  of 
a semester  in  their  perceptions  of  collaboration  as  reported  in  pre-  and  post-surveys.  In  addition,  using  an  assessment 
rubric  we  developed,  we  examined  students’  audience  awareness  as  demonstrated  in  their  writing.  Both  groups  of 
students  were  taught  by  the  same  instructor  and  had  the  same  writing  assignments;  both  classes  incorporated  a 
pedagogy  of  collaborative  learning  to  help  students  develop  a sense  of  belonging  to  a discourse  community;  and 
both  sections  used  networked  computers  as  learning,  writing,  and  communication  tools.  The  distinguishing  variable 
between  the  two  sections  was  the  absence  of  face-to-face  communication  among  students  in  the  online  class.  In  our 
study,  we  posed  the  following  research  questions:  In  a comparison  of  students  in  a on-campus  networked  writing 
class  with  students  in  an  off-campus  online  writing  class,  (a)  Do  students  differ  in  how  they  value  collaboration 
during  the  writing  process?  (b)  Do  students  differ  in  how  they  address  audience  needs  in  their  writing  products?  (c) 
Do  students  differ  in  their  level  of  satisfaction  with  their  learning  experience? 

Our  rationale  for  focusing  on  audience  awareness  was  quite  simple:  One  of  the  hallmarks  of  critical 
thinking  and  thus  of  good  academic  writing  is  the  ability  to  examine  an  issue  from  various  perspectives,  to  take  into 
account  opposing  views,  to  be  aware  that  one  is  writing  not  for  oneself  but  for  an  audience  of  readers  who  have 
multiple  perspectives  and  often  considerable  skepticism  toward  the  writer’s  perspective.  The  best  way  to  increase 
one’s  credibility  with  a skeptical  audience  is  to  acknowledge  readers’  likely  questions,  concerns,  and  objections  and 
to  address  them,  summarizing  opposing  views  fairly,  and  responding  to  those  views  either  through  concession  or 
carefully  crafted  rebuttal  (Rogers,  1961).  Fundamentally,  the  writer’s  obligation  is  not  to  create  barriers  between 
herself  and  the  reader  but  rather  to  build  bridges — to  find  common  ground.  As  beginning  writers,  first-year  college 
students  have  difficulty  doing  this,  largely  because  they  are  locked  in  their  personal  perspectives,  often  viewing  the 
world  in  a dualistic,  “right  and  wrong,”  lens  (Perry,  1970;  Perry,  1985).  Collaboration  during  the  writing  process 
helps  students  expand  their  awareness  of  audience  and  thus,  presumably,  helps  them  improve  this  important  aspect 
of  their  writing. 

Objectives  of  the  freshman  composition  are  to  improve  students’  research  and  argumentative  writing  skills 
and  to  help  students  gain  confidence  in  their  writing  ability.  Peer  response  sessions  on  papers-in-progress  were  an 
important  course  component,  promoting  collaborative  learning  and  heightened  audience  awareness.  To  promote 
collaborative  learning  in  both  the  on-campus  and  the  online  sections,  students  used  FirstClass  software,  a 
communication  and  conferencing  package  that  facilitates  students’  ability  to  collaborate  with  peers  and  to  engage  in 
the  various  stages  of  the  writing  process.  For  the  on-campus  students,  the  computer  classroom  was  equipped  with  24 
networked  PC  workstations  arranged  around  the  perimeter  of  the  classroom,  along  with  an  instructor  workstation 
connected  to  a video  display  projector.  Students  in  the  on-campus  section  (N  = 18)  met  twice  a week  in  the 
networked  classroom.  Students  in  the  online  section  (N  = 15),  on  the  other  hand,  with  the  exception  of  an  initial  and 
final  on-campus  meeting,  completed  all  their  interactions  with  peers  and  the  instructor  online. 

But  whether  students  were  enrolled  in  the  on-campus  or  the  online  class,  they  were  able  to  access  their 
virtual  classroom  space,  including  assignments  and  work-in-progress  by  students  in  the  class,  both  from  home 
computers  and  from  computers  in  open  labs  on  campus.  In  both  classes,  the  instructor  gave  the  students  a brief 
introduction  and  a written  instructional  guide  to  the  technology  itself.  Additionally,  since  this  was  a second-semester 
course,  most  of  the  students  had  had  previous  exposure  to  FirstClass  in  their  first  semester  of  composition  and  were 
thus  familiar  with  the  software  from  the  start  of  the  semester. 

To  determine  demographic  differences  between  the  students  in  the  on-campus  and  the  online  class,  we 
compiled  profiles  of  students  in  the  two  sections  from  survey  questions  (age,  gender,  GPA,  grade  level,  work  load, 
family  status,  technical  expertise,  and  experience)  and  from  the  University  Student  Information  System  (SIS). 
Significance  in  this  study  was  set  at  p = .05. 

Procedure  for  Measuring  Students’  Attitudes  toward  Collaboration 

Besides  providing  demographic  information,  students  in  both  sections  completed  pre-  and  post-surveys 
(Fowler,  1993;  Ehrmann  & Zuniga,  1997)  regarding  their  attitudes  toward  collaboration,  responding  to  questions  on 
a 6-point  Likert-type  scale.  To  increase  face  validity  of  these  items,  we  distributed  the  instrument  to  twelve 
experienced  composition  faculty  in  the  UCCS  English  Department  Writing  Program  who  reviewed  the  items  and 
made  suggestions. 
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Procedure  for  Measuring  Students’  Performance  in  Audience  Awareness 

To  determine  differences  in  demonstrated  level  of  audience  awareness,  the  first  and  final  papers — out-of- 
class,  research-based  argumentative  papers — were  collected  from  all  students  in  both  sections.  For  control  purposes, 
students  were  given  the  same  assignments  for  the  first  and  final  papers.  We  wanted  students  to  write  on  two  topics  at 
a similar  level  of  “strength  of  opinion.”  To  determine  this,  we  polled  the  students  in  the  pre-survey,  asking  them  to 
rank  six  topics  on  a scale  of  one  to  five,  “one”  representing  indifference  to  a topic,  and  “five”  representing  a strongly 
held  position  on  a topic.  The  two  topics  with  the  most  similar  means  were  gun  control  and  capital  punishment.  Not 
only  were  the  means  similar  (gun  control  topic  X = 3.6429,  1 -5  scale;  capital  punishment  X = 3.6667,  1 -5  scale),  but 
students  also  had  relatively  strong  positions  on  these  issues,  so  we  reasoned  that  these  two  topics  would  require 
similar  levels  of  cognitive  challenge  for  students  in  addressing  an  opposing  audience.  The  first  paper  assignment 
instructed  students  to  take  and  support  a position  on  the  issue  of  gun  control,  while  the  final  paper  assignment 
instructed  them  to  take  and  support  a position  on  the  issue  of  the  death  penalty.  In  both  cases,  students  were 
instructed  to  address  an  audience  that  disagreed  with  the  position  they  took  on  the  issue.  The  writer’s  purpose  was  to 
gain  the  readers'  respect,  if  not  their  assent,  for  the  position  argued. 

The  papers  were  coded  using  a random  numerical  coding  system  and  were  assessed  by  three  experienced 
readers  who  first  completed  a "norming"  of  six  of  the  papers,  randomly  selected  (Elbow,  1996).  The  readers 
assessed  the  papers  based  on  a nine-item  rubric,  using  a primary-trait  six-point  criterion-referenced  scale  (Walvoord 
& Anderson,  1998)  that  we  developed.  The  nine  items  include  six  elements  of  audience  awareness  important  in 
argumentative  writing,  the  genre  focus  of  English  141:  purpose,  empirical  support,  logical  appeal,  ethical  appeal, 
emotional  appeal,  and  treatment  of  opposing  views.  These  are  based  on  Aristotle’s  logos,  ethos,  and  pathos,  on 
Toulmin's  (1958)  model  of  informal  reasoning,  and  on  Rogerian  rhetoric  (Rogers,  1961).  We  also  examined  three 
additional  elements  considered  standard  in  essay  assessment:  organization,  syntax,  and  grammar.  Trimble  (2000) 
argues  that  these  additional  elements  do  in  fact  fall  under  the  rubric  of  audience  awareness,  and  that  writing  for  an 
audience  is  less  effective  in  the  absence  of  control  in  these  areas.  Inter-rater  reliability  scores  for  the  rubric  elements 
ranged  from  0.66  to  0.89. 

Results  and  Discussion 

The  online  class  was  significantly  older  (M  =28.36  years)  than  the  on-campus  class  (M  = 20.5,  F = 13.167, 
P < .01),  and  a significantly  higher  portion  of  the  online  students  were  married  (p  < .01).  Although  age  was 
correlated  with  other  demographic  variables,  such  as  the  number  of  dependents,  employment  hours,  and  credit 
hours,  these  other  variables  were  not  significantly  different  between  the  two  sections. 

Online  students  and  on-campus  students  did  not  differ  significantly  in  academic  background.  Students  had 
comparable  grade  point  averages,  TSWE  scores,  SAT-English  scores,  ACT-English  scores,  and  grades  in  English 
131,  the  prerequisite  composition  course  for  English  141.  Students  in  both  sections  were  similarly  comfortable  with 
computer  technology,  Internet  access,  and  CyberClass  usage.  However,  several  interesting  motivational  differences 
between  the  sample  populations  were  evident.  Online  students  ranked  themselves  higher  on  self-discipline  than  did 
on-campus  students,  but  they  planned  to  devote  fewer  hours  studying  for  the  class.  This  difference  is  significant, 
even  when  age  was  used  as  a covariate  (F  = 6.473,  p < .01).  In  addition,  online  and  on-campus  students  had  different 
motivations  for  taking  the  class  that  they  selected.  Online  students  cited  convenience,  while  on-campus  students 
cited  the  good  time  block  as  being  the  primary  reason  for  choosing  the  particular  section. 

Collaboration  Survey  Results 

At  the  beginning  of  the  semester,  students  in  both  sections  held  similar  views  of  collaboration,  with  no 
significant  difference  in  overall  collaboration  scores  between  the  two  sections.  However,  online  students' 
collaboration  scores  were  significantly  lower  at  the  end  of  the  semester  than  they  had  been  at  the  beginning, 
suggesting  a decrease  in  their  valuing  of  collaboration  as  the  semester  progressed.  This  finding  is  in  contrast  to  the 
literature  that  suggests  that  electronic  communication  enhances  a sense  of  community  (Harasim,  1990).  In  our  study, 
online  students  valued  community  less,  believed  less  strongly  that  knowing  other  students  in  the  class  improved 
learning,  and  exhibited  less  preference  for  face-to-face  communication  over  written  communication. 

Factor  analysis  of  the  25  collaboration  survey  items  resulted  in  two  factors,  accounting  for  42.589%  of  the 
variance.  The  reliability  for  this  instrument  was  a = .8910.  Factor  one  (“valuing  peer  feedback  on  work  in  progress”) 
contained  twelve  items,  while  factor  two  (“sense  of  belonging  to  a discourse  community”)  comprised  four  survey 
items.  Using  ANOVA,  we  found  no  significant  differences  on  either  the  pre-  or  the  post-survey  for  factor  one: 
Whether  they  were  enrolled  in  the  online  or  the  on-campus  section,  no  significant  differences  emerged  in  how 
students  valued  peer  feedback  on  work  in  progress  either  on  the  pre-survey  or  the  post-survey.  We  also  calculated 
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the  difference  between  sections  in  the  amount  of  change  in  attitudes  in  factor  one.  While  the  scores  for  the  online 
students  went  down  over  the  course  of  the  semester  (M  = -2.33)  and  the  scores  for  the  on-campus  students  went  up 
(M  = .357),  the  difference  between  groups  in  the  amount  of  change  was  not  significant  (F  = 1 .21 , p < .282). 

Analysis  of  responses  towards  factor  two  items  indicates  that  feeling  connected  was  significantly  less 
important  to  the  learning  experience  for  online  students  than  it  was  for  on-campus  students.  While  the  scores  for  the 
online  students  went  down  over  the  course  of  the  semester  (M  = -1.38)  and  the  scores  for  the  on-campus  students 
went  up  (M  = 2.00),  the  difference  in  the  amount  of  change  was  not  significant  (F  = 2.451,  p < .130). 

Audience  Awareness  Rubric  Results 

In  examining  the  survey  results,  we  were  primarily  interested  in  students’  attitudes  and  self-perceptions.  In 
examining  students’  final  papers,  we  shifted  our  gaze  from  attitudes  to  actual  writing  performance,  focusing 
particularly  on  students’  demonstrated  audience  awareness.  We  wanted  to  see  if  any  differences  emerged  in 
students’  writing  depending  on  whether  students  collaborated  on  their  papers  in  a face-to-face  environment  or 
exclusively  online. 

We  compared  scores  on  the  nine  elements  of  the  audience  awareness  rubric,  on  both  the  first  and  the  final 
papers.  In  spite  of  the  lack  of  face-to-face  collaboration,  online  students  scored  significantly  higher  on  eight  of  the 
rubric  elements  on  the  first  paper  and  on  all  nine  of  the  elements  on  the  final  paper,  as  can  be  seen  in  Table  1 : 

Table  1 

Comparison  of  Means  in  Audience  Awareness  Between  Sections 


First  Paper  M Final  Paper  M 


Rubric 

Element 

Online 

ampus 

( MDif 

Online 

Campus 

MDif 

1. 

Purpose 

13.07 

10.56 

2.51* 

13.60 

10.67 

2.93** 

2. 

Empirical 

evidence 

13.20 

10.44 

2.76* 

14.33 

10.89 

3.44** 

3. 

Logical  appeal 

12.73 

10.33 

2.40* 

14.67 

10.11 

4.56** 

4. 

Ethical  appeal 

12.53 

10.22 

2.31* 

14.47 

10.56 

3.91** 

5. 

Emotional  appeal 

12.73 

10.94 

1.79* 

14.13 

11.44 

2.69** 

6. 

Treatment  of 

Opposing 

Views 

11.07 

9.06 

2.01 

13.20 

10.29 

2.91* 

7. 

Organization 

12.67 

10.89 

1.78* 

14.00 

9.44 

4.56** 

8. 

Syntax 

13.60 

11.06 

2.54* 

14.33 

10.78 

3.55** 

9. 

Grammar 

13.20 

10.89 

2.31* 

14.27 

10.78 

3.49** 

*P  < .05.  **p  < .001 


In  addition,  students  in  the  online  section  showed  a significantly  greater  amount  of  change  in  logical 
appeal,  ethical  appeal,  and  organization  than  did  the  on-campus  students.  From  the  data,  it  would  appear  that 
students  in  the  online  section  developed  better  audience  awareness  skills,  such  as  use  of  logical,  ethical,  and 
emotional  appeals,  and  treatment  of  opposing  views,  as  the  semester  progressed,  while  the  corresponding  on-campus 
students  did  not.  Neither  section  improved  significantly  in  organization,  syntax,  or  grammar.  Scores  on  these  three 
elements  actually  decreased  for  the  on-campus  section,  although  the  decrease  is  not  significant. 

Age  as  a Confounding  Independent  Variable  in  Audience  Awareness  Rubric  Results 

Age  had  a moderate  relationship  to  audience  awareness  scores  on  the  first  paper  (R=  0.452,  p < .01)  and  on 
the  final  paper  (R  = 0.522,  p < .01),  and  a moderate  relationship  to  the  final  course  grade  (R  = 0.437,  p < .05).  When 
age  was  used  as  a covariate,  we  still  found  significant  differences  between  the  two  sections  with  respect  to  ethical 
appeal,  logical  appeal,  and  organization. 

Satisfaction  Level  Results 

Students  in  both  sections  reported  positive  experiences  with  their  respective  classes,  with  similarly 
favorable  evaluations  of  the  instructor,  with  the  on-campus  section  evaluations  being  slightly  more  favorable,  but 
not  statistically  significant.  Rating  for  the  instructor  in  the  on-campus  section  was  an  A,  and  in  the  online  section,  an 
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A-.  One  area  in  which  students  in  the  two  sections  differed  significantly  was  in  their  perceptions  of  the  course 
workload.  Students  in  the  online  section  perceived  the  workload  as  being  more  difficult  (M  = 6.23)  than  did  the  on- 
campus  students  (M  = 5.14),  even  though  the  syllabus,  schedule,  course  assignments,  and  deadlines  were  identical 
for  the  two  classes,  and  the  actual  workload  was  identical.  One  explanation  is  that  online  students  had  significantly 
lower  expectations  of  time  to  be  spent  studying  as  well.  It  could  be  that  the  contrast  between  their  perception  of  the 
workload  and  the  actual  workload  made  the  actual  workload  seem  heavier.  But  in  spite  of  the  fact  that  the  online 
students  believed  that  they  had  a higher  workload  than  students  did  in  the  on-campus  class,  online  students 
overwhelmingly  indicated  that  they  would  prefer  to  take  an  online  class  to  an  on-campus  writing  class. 

Limitations 

The  most  important  limitation  to  this  study  was  the  fact  that  students  self-selected  into  the  networked  and 
online  classes,  and  thus  did  not  represent  a truly  random  sampling  of  populations.  Additionally,  the  small  sample 
size  makes  generalization  speculative.  However,  as  a preliminary  study  of  online  versus  on-campus  writing  classes, 
the  research  reported  in  thisv  paper  was  fruitful  for  us  and  provides  a sound  basis  for  further  exploration  on 
collaboration  and  audience  awareness  in  computer-mediated  freshman  writing  classes. 

Conclusions 

The  results  of  this  study  indicate  that  students  who  enroll  in  an  online  class  have  characteristics  that 
differentiate  them  from  students  who  don’t  select  into  an  online  class.  Another  conclusion  that  we  can  draw  from 
this  study  is  that  online  students  tend  to  be  more  independent  learners,  valuing  collaboration  less  than  do  on-campus 
students.  Additionally,  as  indicated  by  the  higher  scores  in  audience  awareness  on  students’  final  papers  in  the 
online  section,  even  when  the  scores  were  covaried  for  age,  students  can  learn  as  well  in  an  online  class  as  in  an  on- 
campus  class.  Online  interaction  among  students  thus  appears  to  increase  audience  awareness  in  students’ 
argumentative  papers.  This  in  itself  is  a surprising  finding.  We  speculated  that  students  in  the  on-campus  section 
would  have  the  advantage  of  having  online  interaction  along  with  face-to-face  interaction.  Nonetheless,  this 
advantage  did  not  translate  significantly  into  improvement  in  students’  papers  while  improvement  among  the  online 
students  was  significant  in  several  elements  of  audience  awareness.  With  the  increasing  emphasis  on  distance 
learning,  this  study  points  to  a need  for  further  investigation  of  the  pedagogical  implications  of  teaching 
undergraduate  writing  courses  online. 
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Abstract 

This  study  examined  the  effect  of  computer-mediated  collaboration  on  conceptual  change  in  organic 
chemistry.  Students  in  second  semester  organic  chemistry  laboratory  performed  a virtual  experiment  using  air- 
sensitive  alkyl  lithium  reagents.  Students  were  randomly  assigned  to  work  individually  at  a computer  (N  = 21)  or  to 
work  in  peer  dyads , with  one  computer  per  pair  of  students  (N  = 22).  Dyad  pairs  were  also  assigned  randomly. 
Prior  to  treatment  assignment,  all  students  took  a pretest  to  establish  baseline  knowledge  of  organic  concepts  and 
laboratory  technique.  Post  tests  were  administered  immediately  following  treatment  and  one  week  later.  The  results 
showed  that  collaboration  at  the  computer  significantly  improved  conceptual  understanding  of  organic  theory,  but 
did  not  significantly  affect  knowledge  of  laboratory  techniques.  This  may  be  a result  of  co-construction  of 
knowledge,  negotiating  shared  understanding,  and  resolution  of  peer  conflicts  (Crook,  1994;  Tao  & Gunstone, 
1999).  Retention  scores  were  also  higher  for  the  peer  dyads,  but  not  significantly  so. 

Collaboration,  Computers,  and  Conceptual  Change 

While  collaborative  learning  has  been  generally  recognized  as  improving  learning  (Bruffee,  1993) 
(Chickering  & Ehrmann,  1987;  Dunlap  & Grabinger,  1996;  Johnson  & Johnson,  1990;  Sharan,  1980)  in  higher 
education,  it  has  been  underutilized,  particularly  in  the  science  disciplines.  Collaborative  learning  in  organic 
chemistry  has  shown  promising  results  in  the  lecture  as  well  as  in  the  laboratory  (Browne  & Blackburn,  1999; 
Glaser  & Poole,  1999;  Hagen,  2000). 

Collaborative  Learning 

Collaborative  learning  is  defined  by  Tao  and  Gunstone  (1999)  as  “two  (or  more)  students  working  together 
on  a task  that  neither  could  do  on  their  own  prior  to  the  collaborative  engagement”  (p.40).  Damon  and  Phelps  (1989) 
claim  that  collaborative  learning  is  beneficial  for  tasks  requiring  “new  insights,  conceptual  shifts,  and  the 
development  of  deep  knowledge  structure”  (p.  40).  To  Saltiel  (1998),  the  relationship  between  partners  is  as 
important  as  the  knowledge  being  sought  (p.  6).  In  collaborative  partnerships,  students  help  each  other  to  accomplish 
more  than  they  ever  would  have  been  able  to  do  working  individually;  the  partners  have  a shared  goal  or  purpose; 
and  the  partners  experience  mutual  support,  respect,  and  loyalty.  These  three  characteristics  differentiate 
collaborative  partnerships  from  merely  working  together  or  cooperation. 

However,  working  together  does  not  insure  working  effectively.  Damon  and  Phelps  (1989)  describe  two 
characteristics — equality  of  engagement  and  mutuality  of  engagement — that  define  positive  collaborative  peer 
relationships.  For  optimal  peer  learning,  both  partners  should  contribute  relatively  equally  with  both  partners 
discussing  their  ideas  and  opinions  explicitly.  Crook  (1994)  asserts  that  peer  collaboration  offers  three  cognitive 
benefits — articulation,  conflict  and  co-construction.  Articulation  refers  to  explicitly  expressing  latent  ideas.  Conflict 
refers  to  the  processes  that  partners  undergo  to  reconcile  conflicting  and  discrepant  ideas.  Having  resolved  the 
apparent  discrepancies,  students  co-construct  knowledge  based  upon  shared  knowledge,  scaffolding  each  other  to 
greater  understanding  (Tao  & Gunstone,  1997).  The  importance  of  each  of  these  factors  to  the  learning  process  is 
not  clearly  understood.  Dunlap  and  Grabinger  (1996)  claim  that  collaboration  facilitates  learning  by  enabling 
students  to  go  beyond  their  “zones  of  proximal  development”  (p.  79).  Being  able  to  share  with  others  helps  reduce 
risks  and  provides  a better  learning  experience.  Collaboration  also  helps  students  see  how  others  solve  problems  and 
provides  feedback. 

Computer-mediated  Collaborative  Learning 

Collaboration  at  a distance  has  recognizable  benefits.  Computer-mediated  communication  provides 
increased  time  for  reflection  (Berge,  1995;  Fishman,  1997;  Wells,  1993),  greater  equality  between  participants 
(Warschauer,  1997),  greater  participation,  and  improved  work  (Oblinger  & Maruyama,  1996).  Warshnauer  (1997) 
reports  that  computer-mediated  collaboration  produces  cognitive  gains,  due  to  the  dual  attributes  of  reflection  and 
interaction..  While  most  of  the  research  on  computer-mediated  collaboration  has  focused  on  online  interactions 


occurring  either  asynchronously  or  synchronously  in  chat  groups  (Berge,  1995;  Herrmann,  1995;  Koschmann,  1996; 
Koschmann,  Feltovich,  Myers,  & Barrows,  1992),  relatively  little  research  has  examined  the  effect  of  face-to-face 
collaboration  at  the  computer  on  learning.  However,  it  is  clear  that  benefits  can  accrue  from  one-on-one 
communication  in  the  classroom  via  the  computer  (Roschelle,  1992;  Tao  & Gunstone,  1997;  Warschauer,  1997). 
These  benefits  may  include  increased  conceptual  understanding  and  decreased  misconceptions. 

Conceptions  and  Misconceptions 

Far  from  being  a blank  slate  when  students  come  into  the  science  classroom,  students  enter  the  classroom 
with  preconceived  ideas  and  theories,  many  of  which  are  incorrect  (Carter  & Bodner,  1987;  Crosby,  1987;  Driver, 
1989;  Griffiths  & Preston,  1992;  Liggitt-Fox,  1997;  Osborne  & Cosgrove,  1983;  Peterson  & Treagust,  1989;  Strike, 
1983).  One  approach  to  reducing  misconceptions  is  by  confronting  students  with  discrepant  events,  showing  the 
mismatch  between  the  current  conception  and  the  confronting  phenomenon  (Posner,  Strike,  Hewson,  & Gertzog, 
1982;  Strike  & Posner,  1985,  1992).  Computer  laboratory  simulations  can  provide  such  discrepant  events,  improve 
conceptual  understanding  and  thereby  reduce  misconceptions. 

Computer  Simulations 

Numerous  attributes  of  computer  simulations  have  been  identified  from  recent  literature.  Computer 
simulations  can  replace  experiments  that  use  hazardous  materials  (Smith,  Jones,  & Waugh,  1986);  reduce  cost 
(Fletcher,  Hawley,  & Piele,  1990);  replace  experiments  that  occur  too  quickly  or  too  slowly  to  be  done  in  a regular 
laboratory  period  (Herron  & Nurrenbem,  1999);  reduce  cognitive  noise,  so  that  students  can  concentrate  on  the 
concepts  involved  in  the  experiments  (Clariana,  1989);  provide  feedback  to  enhance  conceptual  understanding 
(Chickering  & Ehrmann,  1987);  provide  dynamic  animations  to  emphasize  the  molecular  level  of  chemical  reactions 
(Williamson,  1992;  Williamson  & Abraham,  1995);  allow  rapid  data  collection  (Vining,  2000);  permit  students  to 
generate  and  test  hypotheses  (Jool ingen  & Jong,  1991);  engage  students  with  high  level  of  interactivity  (Grosso, 
1993);  present  a simplified  version  of  reality  by  distilling  abstract  concepts  into  their  most  important  elements 
(White,  1993;  Windschitl  & Andre,  1998;  Zietsman  & Hewson,  1986),  making  abstract  concepts  more  concrete 
(Rieber  & Parmley,  1995);  standardize  instructional  pedagogy,  teaching,  and  content  across  multiple  lab  sections 
(Hilosky,  Sutman,  & Wang,  1997);  actively  engage  students  in  scientific  inquiry  (Grosso,  1993);  reduce  ambiguity 
and  help  identify  cause  and  effect  relationships  in  complex  systems  (Clariana,  1989);  serve  as  pre-laboratory 
preparation  to  aid  understanding  of  the  lab  (Bobbert,  1982);  foster  problem-solving  skills  (Katkanant,  1990;  LaJoie, 
1993);  promote  critical  thinking  skills  (Bonk  & Reynolds,  1997);  help  students  learn  about  the  natural  world  through 
seeing  and  interacting  with  underlying  scientific  models  that  would  not  be  readily  inferred  through  first-hand 
observation  (Krajcik  & Lunetta,  1987);  and  promote  conceptual  change  (White,  1993;  Windschitl  & Andre,  1998). 
From  this  literature,  computer  simulations  clearly  have  the  potential  of  improving  conceptual  understanding  of 
chemical  principles  and  theories  and  teaching  laboratory  skills. 

If  collaboration  and  computer  simulations  each  enhance  conceptual  understanding,  can  learning  from 
computer  simulations  be  improved  by  collaboration  at  the  computer?  The  present  study  was  an  attempt  to  answer 
this  question. 

The  Study 

The  Virtual  Organic  Laboratory  is  a series  of  computer  simulations  designed  to  mirror  a real  organic 
chemistry  laboratory.  In  the  Virtual  Organic  Lab,  students  perform  experiments,  analyze  results  using  spectroscopy 
and  chromatography,  and  write  results  in  their  notebooks.  However,  the  Virtual  Organic  Lab  contains  chemicals  and 
equipment  to  do  experiments  that  would  be  too  costly  or  too  hazardous  to  do  on  a large  scale  in  the  real  laboratory. 
Among  the  virtual  experiments  developed  thus  far  are  oxymercuration-demercuration,  alkyl  lithium  addition,  and 
hydrogenation  using  a Paar  hydrogenator. 

Attributes  of  the  Virtual  Organic  Laboratory 

The  simulations  incorporate  three  characteristics  of  effective  learning:  animation  (Rieber,  1990;  Rieber  & 
Parmley,  1995;  Williamson,  1992)  interactivity  (Borsook  & Higgenbotham-Wheat,  1991),  and  feedback  (Azevedo 
& Bernard,  1995).  Animations  of  reaction  mechanisms  provide  learners  with  step-by-step  pathways,  demonstrating 
lecture  concepts.  Students  can  visualize  electrons  forming  new  bonds  and  bonds  being  broken.  Experimental 
procedures  are  both  animated  and  interactive:  Students  watch  animated  demonstrations  of  laboratory  techniques, 
then  set  up  their  own  equipment  by  dragging  to  the  appropriate  spot.  They  mix  reagents,  conduct  experiments,  and 
isolate  and  purify  products,  mirroring  the  very  processes  they  would  do  in  the  laboratory 


149 

158 


During  the  simulation,  students  receive  continual  feedback  for  both  correct  and  incorrect  choices.  Students 
receive  points  for  each  aspect  of  the  program  that  they  do  correctly,  such  as  correctly  calculating  the  theoretical 
yield  or  remembering  to  take  the  stopper  out  of  the  separatory  funnel  before  draining.  At  the  conclusion  of  the 
simulation  students  receive  a detailed  scorecard  showing  their  performance  in  twenty  categories. 

The  effectiveness  of  the  simulations  in  improving  conceptual  understanding  of  the  reaction  mechanism  and 
improving  technical  understanding  of  laboratory  techniques  was  verified  by  a pilot  project  during  Fall,  1999. 
Students  using  the  simulation  had  significantly  higher  scores  than  students  who  did  not  do  the  simulation  (Xs  = 8.82; 
Xns  = 3.83;  F = 21.45,  p = 0.001).  However,  these  scores  were  still  unacceptably  low.  In  an  effort  to  improve 
learning  outcomes,  we  decided  to  investigate  how  collaboration  at  the  computer  affects  conceptual  understanding. 

Design  of  the  Study 

During  Spring,  2000,  students  in  each  organic  chemistry  laboratory  section  were  randomly  divided  into  two 
treatment  groups  with  one  group  of  students  (N  = 21)  working  individually  on  the  computer  simulation  and  the  other 
group  of  students  (N  = 22)  randomly  paired  into  peer  dyads.  The  dyad  pairs  worked  collaboratively  on  one  computer 
simulation.  Prior  to  treatment  assignment,  students  took  a pretest  to  assess  extant  knowledge  and  understanding  of 
organic  reaction  mechanisms,  inert  gas  chemistry,  alkyl  lithium  reactions,  and  other  organic  chemistry  techniques. 
The  students  also  completed  a survey  to  assess  their  attitudes  about  collaboration  and  the  laboratory.  After  doing  the 
simulation,  students  completed  a post  test  and  attitudinal  survey. 

Results  of  the  Study 

Prior  to  treatment,  the  two  groups  of  students  were  comparable,  with  no  significant  academic  or  attitudinal 
differences  between  the  two  groups.  Both  groups  had  similar  pretest  scores  (Xc  = 29.9%  and  Xj  = 36.5%).  Grades  in 
the  previous  semester  lecture  were  similar  (Xc  = 3.49  and  Xj  = 3.59),  as  were  the  grades  received  in  the  previous 
semester  organic  chemistry  lab  (Xc  = 3.56  and  Xi  = 3.51,  out  of  4.00).  Both  groups  held  similar  views  about  the 
importance  of  collaboration  (Xc  = 3.52  and  Xs  = 3.52,  out  of  5.00)  and  organic  chemistry  laboratory  (Xc  = 2.55  and 
Xj  = 2.52,  out  of  5.00). 

After  treatment,  however,  there  were  some  significant  differences  in  performance  on  the  post  test,  with 
students  in  peer  dyads  significantly  outperforming  students  who  worked  individually  ( = 63.2%,  Xj  = 54.9%,  F = 
7.920,  p < 0.01).  Peer  dyads  significantly  outperformed  individual  users  on  conceptual  questions  relating  to  the 
reaction  mechanism  (Xc  = 83.0%  and  Xj  = 64.9%,  p < 0.01).  The  students  in  peer  dyads  also  scored  higher  on 
questions  relating  to  laboratory  techniques  and  instrumentation;  however  the  differences  were  not  significant  (Xc  = 
59.6%,  Xj  = 54.0%,  p > 0.05).  The  results  imply  that  understanding  organic  theory  requires  different  learning 
strategies  than  does  laboratory  technique  understanding.  Conflict  (Cook  & Cook,  1998)  and  peer  discussion 
(McMillan,  1997;  Winiecki,  1999)  may  aid  student  understanding  of  complex  mechanisms.  Collaboration  at  the 
computer  facilitates  this  type  of  peer  interaction.  On  the  other  hand,  gaining  laboratory  expertise  and  understanding 
the  practicalities  and  theory  behind  laboratory  techniques  and  instrumentation  may  not  benefit  as  much  from 
discussion  and  the  learning  advantages  of  peer  discussion  may  be  partially  compensated  by  individual  use  of  the 
simulation. 

A retention  test  was  administered  two  weeks  after  the  simulation  laboratory  sessions.  Again,  students  in 
peer  dyads  outperformed  students  who  worked  individually  on  both  the  mechanism  and  laboratory  techniques 
sections  of  the  retention  test.  However,  due  to  the  small  number  of  questions  in  both  sections,  the  differences 
between  the  groups  were  not  significant  and  no  generalizations  can  be  drawn. 

Interestingly,  although  pretest  scores  were  moderately  correlated  to  grade  in  first  semester  organic 
chemistry  class,  post  test  scores  for  both  groups  were  not  significantly  correlated  to  grade  students  received  in  the 
first  semester  organic  chemistry  class.  This  result  suggests  that  computer  simulations  improve  scores  for  lower 
performing  students.  This  result  contradicts  the  research  of  Vaidyanathan  and  Rochford  (1998)  who  found  that 
students  who  performed  well  on  lecture  exams  also  performed  well  in  computer  simulations. 

Attitudinal  Results 

There  were  no  significant  differences  in  attitudes  towards  the  computer  simulation:  (Xc  = 18.4  and  Xj 
=19.4,  F = 0.870,  p = 0.357)).  Both  groups  of  students  thought  they  had  learned  well  from  the  computer  simulation, 
with  the  majority  of  students  agreeing  that  the  Virtual  Organic  Laboratory  simulation  had  helped  them  understand 
the  mechanism  and  reactivity  of  organic  lithium  reactions  (69.7%),  the  techniques  of  conducting  air-sensitive 
reactions  (95.3%),  and  the  theory  and  practice  of  extractions  (81.4%).  The  percentages  refer  to  the  percent  of 
students  strongly  agreeing  with  the  indicated  statements.  Both  groups  were  also  very  positive  about  using  the 
computer  simulations,  with  93.0%  of  all  students  strongly  agreeing  that  the  Virtual  Organic  Laboratory  simulation 
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was  fun  and  85.7%  of  the  students  wanting  to  use  the  simulations  used  for  other  experiments.  Over  half  of  the 
students  (62.8%)  indicated  that  they  felt  as  though  they  had  carried  out  a real  laboratory  experiment  and  69.8% 
believed  that  doing  the  simulation  was  a better  learning  experience  than  doing  an  actual  laboratory  experiment. 
Factor  analysis  was  performed  to  understand  students’  preference  for  computer  simulations  over  laboratory 
experiments.  Two  factors  were  isolated  that  explained  students’  satisfaction  with  computer  simulations.  Factor  one 
(a  = 0.8696)  related  to  confidence:  Doing  the  simulation  improved  students’  confidence  in  doing  lab  work.  Factor 
two  (a  = 0.8932)  related  to  time:  Students  thought  that  doing  the  simulation  made  learning  more  efficient  and  saved 
time.  There  were  no  significant  differences  between  the  collaborative  and  non-collaborative  groups  in  either  factor. 

Conclusions 

Two  general  conclusions  can  be  drawn  from  this  research.  First,  students  who  work  collaboratively  at  the 
computer  gain  greater  understanding  of  mechanisms  and  reactivity  than  students  who  work  alone.  Working  together 
gives  students  an  opportunity  for  peer  conflict,  discussion,  and  co-construction  of  knowledge  (Crook,  1994). 
Students  working  in  pair  dyads  must  verbalize  their  existing  conceptions  as  they  attempt  to  make  sense  and 
incorporate  new  and  discrepant  information.  This  requires  that  students  summarize  their  existing  ideas  in  order  to 
share.  The  act  of  verbalization  may  also  serve  as  a catalyst  to  recognize  erroneous  ideas  and  perhaps  modify 
concepts.  Students  who  work  alone  do  not  have  a peer  to  sound  new  ideas  and  are  not  challenged  to  explain  existing 
conceptions.  As  a result,  students  who  worked  individually  may  not  have  as  much  opportunity  to  recognize 
inadequate  conceptions  or  to  give  up  alternative  concepts. 

A second  conclusion  is  that  computer  simulations  enhance  students’  enjoyment  of  the  laboratory. 
Regardless  of  whether  students  worked  individually  or  collaboratively,  students  felt  cognitively  engaged  in  doing 
the  simulation  and  felt  as  though  they  had  learned  a substantial  amount  of  the  theory  and  practice  of  working  with 
alkyl  lithium  reagents.  This  finding  is  corroborated  by  numerous  other  studies  (Escalada  & Zollman,  1997;  Kulik, 
1994;  Strauss  & Kinzie,  1994;  Zirkel  & Zirkel,  1997).  Students  felt  that  they  learned  more  efficiently  from  the 
computer  simulation  and  felt  confident  that  they  had  learned  the  techniques  well. 

Most  students  indicated  that  they  prefer  computer  simulations  to  hands-on  laboratory  work.  This  may  be 
due  to  the  novelty  effect  of  doing  computer  simulations  and  the  perception  of  computer  simulations  as  requiring  less 
time  than  performing  a real  experiment.  However,  other  six  other  attributes  of  the  computer  simulation  contribute  to 
their  preference  for  computer  simulation  over  traditional  wet  laboratory  experiments:  (a)  Students  aren’t  limited  to 
safe,  inexpensive,  and  available  chemicals,  but  can  do  a wide  range  of  reactions,  illustrating  and  supporting  concepts 
learned  in  lecture,  (b)  Students  can  repeat  an  experiment  as  many  times  as  necessary  to  understand  the  concepts, 
mechanism,  or  procedural  attributes  of  the  experiment,  (c)  Students  can  gain  experience  using  expensive  equipment 
or  complex  spectroscopic  techniques,  (d)  Students  can  practice  a technique  before  they  have  to  do  it  in  lab,  thereby 
improving  their  confidence  in  doing  lab  work,  (e)  Students  can  make  mistakes  without  risking  time,  money,  or 
safety,  (f)  Students  can  get  rapid,  almost  instant,  feedback,  thereby  improving  learning.  In  addition  to  these  cognitive 
benefits,  the  Virtual  Organic  Laboratory  simulations  offer  another  practical  benefit:  Students  can  do  the  lab  any  time 
of  the  day,  from  anywhere  in  the  world. 

While  computer  simulations  will  never  replace  traditional  laboratory  experiments,  laboratory  simulations 
may  be  the  answer  to  the  increasing  need  to  deliver  a pedagogically  sound  laboratory  experience  in  distance  science 
courses.  Regardless  of  whether  students  are  at  a distance  or  face  to  face,  computer  simulations  can  enhance  learning, 
particularly  if  students  are  allowed  to  collaborate,  either  at  the  computer  or  in  cyberspace. 
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What  and  How  Do  Designers  Design?  A Theory  of  Design  Structure 
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Abstract 

A question  I always  ask  my  Instructional  Technology  students  at  Utah  State  University  is,  “What  do 
instructional  designers  design ?”  We  have  had  interesting  discussions  on  this  question,  and  I try  to  revisit  the 
question  at  several  points  throughout  all  of  my  classes.  I find  that  the  students ' perceptions  of  what  instructional 
designers  design  changes  over  time.  This  is  no  doubt  a product  of  the  faculty's  teaching,  but  it  also  represents  a 
personal  commitment  that  the  student  makes.  What  the  student  commits  to  is  what  I would  like  to  talk  about.  My 
thesis  will  be  that  it  is  a commitment  to  a particular  layer  of  the  evolving  instructional  design.  I will  talk  about  the 
layering  of  instructional  designs  and  the  implications  for  both  teaching  and  practicing  instructional  design. 

“The  Centrisms” 

Here  are  some  of  the  phases  I see  students  evolving  through  as  they  mature  in  their  theoretic  and  practical 
knowledge: 

Media-centrism.  Media-centric  designs  place  great  emphasis  on  the  constructs  related  to  the  instructional  medium. 
The  technology  itself  holds  great  attraction  for  new  designers,  They  often  construct  their  designs  in  the  vocabulary 
of  the  medium  rather  than  seeing  the  medium  as  a plastic  and  preferably  invisible  channel  for  learning  interaction 
(See  Norman,  1988;  1999).  We  are  currently  experiencing  a wave  of  new  media-centric  designers  due  to  the 
accessibility  of  powerful  multimedia  tools  and  large  numbers  of  designers  “assigned  into”  computer-based  and 
Web-based  training  design.  Most  of  these  designers  speak  in  terms  of  the  medium’s  constructs  (the  "page,"  the 
"hyperlink,",  the  "site,"  etc.)  as  the  major  design  building  blocks.  Many  struggle  as  they  attempt  to  apply  inadequate 
thought  tools  to  complex  design  problems. 

Message-centrism.  Realizing  that  media  design  building  blocks  do  not  automatically  lead  to  effective  designs,  most 
designers  begin  to  concentrate  on  "telling  the  message  better"  in  order  to  "get  the  idea  across"  or  "make  it  stick." 
This  is  a phase  I call  message-centrism.  Message-centric  design  places  primary  importance  on  message-related 
constructs  and  employs  media  constructs — main  idea,  explanation,  line  of  argument,  dramatization,  etc. — 
secondarily,  according  to  the  demands  of  the  message.  The  media  constructs  are  used,  but  they  are  used  to  serve  the 
needs  of  better  messaging.  Better  message  “telling”  means  different  things  to  different  designers:  providing  better 
illustrations,  using  animations,  wording  the  message  differently,  using  analogies,  or  focusing  learner  attention  using 
attention-focusing  questions,  emphasis  marks,  repetition,  or  increased  "interactivity." 

Strategy-centrism.  Message-centrism  is  normally  followed  by  a recognition  of  underlying  structural  similarities 
within  messages  and  interactions  that  cross  subject-matter  boundaries  and  that  have  important  instructional 
implications.  This  leads  to  a new  viewpoint  I call  strategy-centric  design  thinking.  Strategy-centrism  considers  the 
structured  plan  of  messaging  and  interaction  as  a main  source  of  instructional  effectiveness.  Therefore,  the 
designer's  first  attention  is  to  strategic  constructs  that  are  appropriate  to  instruction  in  categorized  varieties  of 
learning.  Strategy-centric  design  can  be  viewed  as  the  use  of  rules  to  governing  the  delivery  of  compartmentalized 
information  and  interaction  elements  (Gagne,  1985;  Merrill,  1994).  This  can  be  a very  useful  conception  for  both  the 
designer  and  the  learner,  and  structured  strategy  is  an  important  key  to  logic  templating  and  design  automation. 

Model-centrism.  Whereas  strategy  centrism  permits  the  use  of  instructional  experts  (Zhang,  Gibbons,  & Merrill, 
1997),  it  does  not  lead  the  designer  to  design  interactive  micro-worlds  in  which  instruction  can  take  place  through 
problem  solving.  This  realization  leads  to  model-centered  design  thinking.  Model  centering  encourages  the  designer 
to  think  first  in  terms  of  the  system  and  model  constructs  that  lie  at  the  base  of  subject-matter  knowledge.  The 
designer  therefore  gives  first  consideration  to  identifying,  capturing,  and  representing  in  interactive  form  the 
substance  these  constructs.  Then  to  this  base  of  design  is  added  strategy,  message,  and  media  constructs.  Model- 
centrism  is  the  common  thread  running  through  virtually  all  new-paradigm  instructional  approaches  (for  a review, 
see  Gibbons  & Fairweather,  2000).  Many  current  researchers  consider  learning  to  be  a problem-solving  activity 
(Anderson,  1993;  Brown  & Palincsar,  1989;  Schank,  1994;  VanLehn,  1993).  If  this  view  is  correct,  then  the  designer 
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must  also  give  first  preference  to  decisions  about  the  problems  the  learner  will  be  asked  to  solve.  A model-centered 
view  prescribes  instructional  augmentations  that  support  problem  solving  in  the  form  of  coaching  and  feedback 
systems,  representation  systems,  control  systems,  scope  dynamics,  and  embedded  didactics  (see  Gibbons, 
Fairweather,  Anderson,  & Merrill,  1997). 

These  phases  in  the  maturation  of  design  thinking  tend  to  be  encountered  by  new  designers  in  the  same 
order,  and  one  could  make  the  argument  that  these  phases  describe  the  history  of  research  interests  in  the  field  of 
instructional  technology  as  a whole.  A good  place  to  see  this  trend  in  cross-section  is  the  articles  in  the  Annual 
Review  of  Psychology’  beginning  with  the  review  by  Lumsdaine  and  May  (1965)  and  progressing  through  subsequent 
chapters  by  Anderson  (1967);  Gagne  & Rohwer  (1969);  Glaser  & Resnick  (1972);  McKeachie  (1974);  Wittrock  & 
Lumsdaine  (1977);  Resnick  (1981);  Gagne  & Dick  (1983);  Pintrich,  Cross,  Kozma  & McKeachie  (1986);  Snow  & 
Swanson  (1992);  Voss,  Wiley  & Carretero  (1995);  Sandoval  (1995);  VanLehn  (1996);  Carroll  (1997);  Palincsar 
(1998);  and  Medin,  Lynch  & Solomon  (2000). 

Roots  of  the  “Centrisms” 

I am  interested  in  this  paper  in  exploring  the  roots  of  this  progression.  Important  clues  can  be  found  in 
design  areas  outside  of  instructional  design.  A provocative  statement  on  design  structure  is  given  by  Brand  (1994)  in 
a description  of  how  buildings  are  seen  by  architects  and  structural  engineers.  Brand  begins  by  stating  that  architects 
see  a building  as  a system  of  layers  rather  than  as  a unitary  designed  entity.  He  names  six  general  layers,  illustrated 
in  Figure  1 and  described  below  in  his  own  words: 

• SITE  - This  is  the  geographical  setting,  the  urban  location,  and  the  legally  defined  lot,  whose  boundaries  and 
context  outlast  generations  of  ephemeral  buildings.  “Site  is  eternal,”  Duffy  agrees. 

• STRUCTURE  - The  foundation  and  load-bearing  elements  are  perilous  and  expensive  to  change,  so  people 
don’t.  These  are  the  building.  Structural  life  ranges  from  30  to  300  years  (but  few  buildings  make  it  past  60, 
for  other  reasons). 


Figure  L Layers  of  building  design . 

• SKIN  - Exterior  surfaces  now  change  every  20  years  or  so,  to  keep  with  fashion  and  technology,  or  for 
wholesale  repair.  Recent  focus  on  energy  costs  has  led  to  reengineered  Skins  that  are  air-tight  and  better 
insulated. 

• SERVICES  - These  are  the  working  guts  of  a building:  communications  wiring,  electrical  wiring,  plumbing, 
sprinkler  system,  HVAC  (heating,  ventilating,  air  conditioning),  and  moving  parts  like  elevators  and  escalators. 
They  wear  out  or  obsolesce  every  7 to  15  years.  Many  buildings  are  demolished  early  if  their  outdated  systems 
are  too  deeply  embedded  to  replace  easily. 

• SPACE  PLAN  - The  interior  layout — where  walls,  ceilings,  floors,  and  doors  go.  Turbulent  commercial  space 
can  change  every  3 years  or  so;  exceptionally  quiet  homes  might  wait  30  years. 

• STUFF  - Chairs,  desks,  phones,  pictures,  kitchen  appliances,  lamps,  hair  brushes;  all  the  things  that  twitch 
around  daily  to  monthly.  Furniture  is  called  mobilia  in  Italian  for  good  reason,  (p.  13) 
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Brand  points  out  some  important  implications  of  the  layered  view  of  design: 

1 . That  layers  of  a design  age  at  different  rates 

2.  That  layers  must  be  replaced  or  modified  on  different  time  schedules 

3.  That  the  layers  must  be  articulated  with  each  other  somehow 

4.  That  designs  should  provide  for  articulation  in  such  a way  that  change  to  one  layer  entails  minimum  disruption 
to  the  others. 

In  work  for  the  Center  for  Human-Systems  Simulation,  my  colleagues  Jon  Nelson  and  Bob  Richards  and  I have 
applied  Brand’s  ideas  to  instructional  design  (Gibbons,  Nelson  & Richards,  2000).  We  have  found  that  instructional 
designs  can  indeed  be  conceived  of  as  multiple  layers  of  decision  making  with  respect  to  different  sets  of  design 
constructs,  and  we  find  a rough  correspondence  between  the  layers  and  the  phases  of  designer  thinking  already 
described.  Gibbons,  Lawless,  Anderson  and  Duffin  (in  press)  show  how  layers  of  a design  are  compressed  at  a 
“convergence  zone”  with  tool  constructs  that  give  them  real  existence  and  embody  them  in  a product. 

Tables  1 through  7 at  the  end  of  this  paper  summarize  what  we  think  are  the  important  layers  of  an 
instructional  design:  model/content,  strategy,  control,  message,  representation,  media-logic,  and  management.  Each 
layer  is  characterized  in  the  tables  by  the  following: 

• A set  of  design  goals  unique  to  the  layer 

• A set  of  design  constructs  unique  to  the  layer 

• A set  of  theoretic  principles  for  the  selection  and  use  of  design  constructs 

• A set  of  design  and  development  tools 

• A set  of  specialized  design  processes 

In  addition,  a layer  often  corresponds  with  a set  of  specialized  design  skills  with  its  own  lore,  design 
heuristics,  technical  data,  measurements,  algorithms,  and  practical  considerations.  The  boundaries  of  these  skills 
over  time  tend  to  harden  into  lines  of  labor  division,  especially  as  technical  sophistication  of  tools  and  techniques 
increases. 

More  detailed  principles  of  design  layering  are  outlined  in  Gibbons,  Nelson,  and  Richards  (2000).  The 
purpose  of  the  present  paper  is  to  show  how  design  layering  influences  the  designer’s  thinking  and  allows  it  to 
change  over  time  into  entirely  new  ways  of  approaching  the  design  task.  The  media-,  message-,  strategy-,  and 
model-centric  phases  designers  experience  can  be  explained  as  the  necessary  focus  of  the  designer  first  and  foremost 
on  a particular  layer  of  the  design.  That  is,  the  designer  enters  the  design  at  the  layer  most  important  to  the  design  or 
with  which  the  designer  is  most  familiar  and  comfortable. 

Media-centric  designers  do  not  ignore  decisions  related  to  other  layers,  but  because  they  may  not  yet  be 
fully  acquainted  with  the  principles  of  design  at  other  layers,  they  naturally  think  in  terms  of  the  structures  they  do 
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know  or  can  acquire  most  rapidly — media  structures.  As  designers  become  aware  of  principles  at  other  layers 
through  experience  and  the  evaluation  of  their  own  designs,  focus  can  shift  to  the  constructs  of  the  different  layers: 
message  structurings,  strategy  structurings,  and  model  and  content  structurings.  Each  step  of  the  progression  in  turn 
gives  the  designer  a new  set  of  constructs  and  structuring  principles  to  which  to  give  the  most  attention,  with  other 
layers  of  the  design  being  determined  secondarily,  but  not  ignored. 

Is  there  a “right”  layer  priority  in  designs?  Should  designers  always  be  counseled  to  enter  the  design  task 
with  a particular  layer  in  mind?  It  is  not  possible  to  say  that,  because  design  tasks  most  often  come  with  constraints 
attached,  and  one  of  those  constraints  may  predetermine  a primary  focus  on  a layer.  An  assignment  to  create  a set  of 
videotapes  will  lead  the  designer  to  pay  first  and  last  attention  to  the  media-logic  and  representation  layers,  and  other 
layers  are  forced  to  comply  with  the  constraint  within  the  limits  of  the  designer’s  ingenuity. 

Conclusion 

The  design  layering  concept  has  many  implications.  In  this  paper  I have  explored  one  of  them  that  explains 
the  maturation  in  designer  thinking  over  time.  In  order  to  move  to  a new  perspective  of  design  it  is  not  necessary  to 
leave  older  views  behind.  The  new  principles  added  as  the  designer  becomes  knowledgeable  about  each  new  layer 
adds  to  the  designer’s  range  and  to  the  sophistication  of  the  designs  that  are  possible.  Further  consideration  of  the 
layering  concept  will  expand  our  ability  to  communicate  designs  in  richer  detail,  achieve  more  sophisticated  designs, 
and  add  to  our  understanding  of  the  design  process  itself. 
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Attachment  A. 

Design  Layering  Principle: 
Application  to  Instructional  Design 
From  Gibbons,  Nelson  & Richards  (2000) 


Table  7:  Model/Content  Layer  Description 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  define  the  units  of  content  segmentation 
To  define  the  method  of  content  capture 
To  gather  content  elements 
To  articulate  content  structures: 

With  the  Strategy  layer 
With  the  Control  layer 
With  the  Message  layer 
With  the  Representation  layer 
With  the  Logic  layer 
With  the  Management  layer 

(Incomplete  sample  list) 
Model 
Relation 
Production  rule 
Working  Memory  Element 
Proposition 
Fact 
Concept 
Rule 
Principle 
Task 

Task  grouping 
Theme 
Topic 
Main  idea 

Semantic  relationship 
Chapter 

Design  Processes:  Task  Analysis,  Cognitive  Task  Analysis,  Rule  Analysis,  Content  Analysis,  Concept 
Mapping 

Design/Production  Tools:  Data  base  software,  Analysis  software 

Table  2:  Strategy/ Event  Layer  Description 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  define  event  structures  (time  structures) 
To  define  event  hierarchies 
To  define  rules  for  event  generation 
To  articulate  strategy  structures: 

With  the  Content  layer 
With  the  Control  layer 
With  the  Message  layer 
With  the  Representation  layer 
With  the  Logic  layer 
With  the  Management  layer 

(Incomplete  sample  list) 
Problem 

Information  event 
Interaction  event 
Exercise 

Instructional  period 
Discovery  challenge 
Unit 
Lesson 

Strategy  component 
Argument 
Argument  support 

Design  Processes:  Strategy  planning,  Problem  planning,  Challenge  formation,  Activity  planning,  Exercise 
design 

Design/Production  Tools:  Data  base  software 
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Table  3:  Control  Layer  Description 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  define  the  set  of  possible  user  actions 

(Incomplete  sample  list) 

To  define  the  rules  of  control  availability 

Menu  item 

To  define  the  rules  for  control  action 

Administrative  control 

To  define  the  rules/processes  for  response 

Strategy  control 

recognition,  parsing,  and  judging 

Message  control 

To  articulate  control  structures: 

Representation  control 

With  the  Content  layer 

Logic  control 

With  the  Strategy  layer 

Content  control 

With  the  Message  layer 

Forward,  Back 

With  the  Representation  layer 

Play,  FF,  FR,  Stop,  Pause 

With  the  Logic  layer 
With  the  Management  layer 

Exit,  Quit 

Design  Processes:  Flow  planning,  Control  walk-through,  Diagramming 

Design/Production  Tools:  Flowcharting,  GUI-logic  construction  authoring  systems 

Table  4:  Message  Layer  Description 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  define  message  types 
To  define  message  composition  by  type 
To  define  rules  for  message  generation 
To  articulate  message  structures: 

With  the  Content  layer 
With  the  Strategy  layer 
With  the  Control  layer 
With  the  Representation  layer 
With  the  Logic  layer 
With  the  Management  layer 

(Incomplete  sample  list) 

Main  idea 

Example 

Non-Example 

Discussion  block 

Commentary 

Advance  organizer 

Primitive  message  clement 

Spatial  relationship 

Temporal  relationship 

Causal  relationship 

Hierarchical  relationship 

Explanation 

Stem 

Distractor 
Response  request 
Transition  message 
Goal  statement 
Directions 
“Resource” 

Database  entry 
Coaching  message 
Feedback  message 
Hint 

Design  Processes:  Message  design,  Strongly  related  to  Strategy  design 

Design/Production  Tools:  Timeline-building  tools,  Flow  diagrams 
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Table  5:  Representation  Layer  Description 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  select  media 

(Incomplete  sample  list) 

To  define  media  channels 

Background 

To  define  channel  synchronizations 

Resource  file  (audio,  video) 

To  define  representation  structures  by  type 

Resource  file  (BMP,  JPG,  GIF,  MPG) 

To  select  representation  production  tools 

Headline,  Body 

To  match  production  tool  structures 

Placeholder 

To  define  rules  display  structure 

3-D  object 

To  define  rules  for  display  generation 

Rendering 

To  define  rules  for  structure  generation 

Animation 

To  define  rules  for  display  management 

Tag  parameter 

To  articulate  representation  structures: 

Sprite 

With  the  Content  layer 

Control  icon 

With  the  Strategy  layer 

Layer  (e.g.,  Photoshop,  Dreamweaver) 

With  the  Control  layer 
With  the  message  layer 
With  the  Logic  layer 
With  the  Management  layer 

Design  Processes:  Display  design,  Formatting,  Display  event  sequencing,  Media  channel  synchronization, 

Media  channel  assignment 

Design/Production  Tools:  All  content/resource  production  tools  for  all  media,  All  layout  or  formatting 

tools  for  all  media,  Display  managers 

Table  6:  Logic  Layer  Description 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  define  media-logic  structures  by  type 

(Incomplete  sample  list) 

To  define  rules  to  apply  logic  structures 

Display 

To  select  logic  construction  tools 

Branch 

To  define  segmentation/packaging  plan 

Program 

To  define  logic  distribution  plan  (time) 

Command 

To  articulate  logic  structures: 

Procedure 

With  the  Content  layer 

Program  object 

With  the  Strategy  layer 

Applet 

With  the  Control  layer 

Application 

With  the  Message  layer 

Book,  object 

With  the  Representation  layer 

Movie,  stage,  actor 

With  the  Management  layer 

Object,  Method,  Data 

Site,  Page 

Design  Processes:  Program  design,  Program  construction 

Design/Production  Tools:  All  logic  production  tools,  Modeling  languages  (e.g.,  UML) 
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Table  7:  Management  Layer  Descriptive 


Layer  Design  Goals 

Common  Layer  Design  Constructs 

To  define  session  control  rules/procedures 
To  define  the  rules  for  initiative  sharing 
To  define  transition  between  events 
To  define  record  keeping  and  recording 
To  define  variable-keeping  and  use 
To  define  outside  communications: 

Host,  Peer,  Net,  Libraries,  Databases 
To  define  data  reporting: 

Learner,  Instructor,  System 
To  plan  security/privacy  policy/provisions 
To  plan  evaluation  activities 
To  plan  implementation  activities 
To  plan  management  activities 
To  articulate  management  structures: 

With  the  Content  layer 
With  the  Strategy  layer 
With  the  Control  layer 
With  the  Message  layer 
With  the  Representation  layer 
With  the  Logic  layer 

(Incomplete  sample  list) 
Menu 
Record 
Variable 
Database  entry 

Design  Processes:  Management  planning,  Implementation  planning,  Evaluation  planning 

Design/Production  Tools:  Data  base  software 
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Abstract 

This  paper  proposes  a model  for  teaching  and  learning  complex  thinking  skills  developed  from  a synthesis 
of  theories  and  research.  The  model  functions  like  a concept  map  or  a graphic  organizer  with  five  major 
components:  (1)  presence  of  complex  authentic  life  situations  within  a context ; (2)  activation  and  execution  of 
complex  thinking  skills ; (3)  development  of  interactive  prerequisites  of  content , simple  thinking  skills,  and 
dispositions  and  habits ; (4)  inclusion  of  connecting  networks  and  operations  (linkages,  schemata,  and  scaffolding) 
to  bridge  complex  thinking  skills  with  interactive  prerequisites;  and  (5)  targeted  teaching  and  learning  strategies. 
Terms  associated  with  thinking  and  learning  guided  the  manual  and  electronic  search  through  the  Internet  and  on- 
line library  files. 

Background 

There  are  known  instructional  strategies  to  support  the  learning  of  different  types  of  knowledge  and  skills 
(Merrill,  Drake,  Lacy,  Pratt,  & the  ID2  Research  Group,  1996),  and  for  the  most  part,  these  were  identified  by 
observing  the  conditions  of  learning  across  six  decades  of  study.  Classifications  of  learning  outcomes  tend  to 
progress  from  simple  to  complex  (Bloom,  1956;  Briggs  & Wager,  1981;  Bruner,  1990;  Clarke,  1990;  Dewey,  1933; 
Gagne,  1985,  1989;  Gagne  & Briggs,  1974;  Gagne,  Briggs,  & Wager,  1988;  Costa,  1990;  Cotton,  1997;  Glaser, 
1941;  Guilford  cited  in  Crowl  et  al,  1997;  Haladyna,  1997;  Jonassen,  n.d.;  Marzano,  1993,  1994;  Marzano,  et  al, 
1992;  Marzano,  et  al,  1988;  Merrill,  1987;  Merrill  et  al,  1996;  McREL,  1997;  Piaget  cited  in  Crowl  et  al,  1997; 
Sternberg,  1998;  Sternberg  & Davidson,  1995;  Sugrue,  1995;  Vygotsky  cited  in  Crowl  et  al,  1997). 

Many  publications  cite  the  distinction  between  lower  order  and  higher  order  thinking  skills  (Arter  & 
Salmon,  1987;  Camine,  1993;  Clarke,  1990;  Ennis,  1989;  Fogarty  & McTighe,  1993;  Crowl  et  al,  1997;  Kauchak  & 
Eggen,  1998;  Kirby  & Kuykendall,  1991;  Lewis  & Smith,  1993;  McDavitt,  1993;  McGregor,  1993;  Paul  & Nosich, 
1992;  Weisberg,  1995;  Young,  1997).  Some  refer  to  tasks  requiring  increased  levels  of  processing  (cognitive: 
classifications,  rule  or  procedural  executions)  and  others  to  tasks  demanding  high  levels  of  processing  (constructive: 
heuristic  problem  solving,  personal  selection  and  monitoring  of  cognitive  strategies)  (Ertmer  & Newby,  1993). 

Yet,  the  definition  of  complex  thinking  skills  has  been  referred  to  as  a conceptual  swamp  (Cuban  cited  in 
Lewis  & Smith,  1993,  p.  1)  and  ...a  century  old  problem  for  which  there  is  no  well-established  taxonomy  or 
typology  (Haladyna,  1997,  p.  32).  What  is  unknown  also  has  been  described  as  follows. 

...While  advanced  knowledge,  higher  order  thinking,  problem  solving,  and  transfer  of  learning 
evoke  common  associations  and  expectations  in  most  of  us,  there  remains  an  operational 
inexactitude  in  these  constructs. .these  learning  outcomes  can  best  be  operationalized  and 
predicted  by  assessing  and  understanding  learners ' mental  models  of  the  problem  or  content 
domain  being  learned.... 

Jonassen,  n.d.,  p.  1 

A Model  for  Teaching  and  Learning  Strategies 

Figure  1 represents  a synthesis  from  theories  of  learning  and  research  associated  with  complex  thinking 
skills.  This  model  functions  like  a concept  map  or  graphic  organizer  (Ausubel,  1968;  Clarke,  1990;  Kealy,  2000; 
Jonassen,  1996;  NCREL,  1988;  Novak,  1990;  Plotnick,1997).  It  shows  components,  linkages,  relationships,  and 
interactions  for  teaching  and  learning  complex  thinking  skills.  The  narrative  further  explains  the  relationships  among 
the  five  components:  (1)  presence  of  complex  authentic  life  situations  within  a context;  (2)  activation  and  execution 
of  complex  thinking  skills;  (3)  development  of  interactive  prerequisites  of  content,  simple  thinking  skills,  and 
dispositions  and  habits;  (4)  inclusion  of  connecting  networks  and  operations  (linkages,  schemata,  and  scaffolding)  to 
bridge  complex  thinking  skills  with  interactive  prerequisites;  and  (5)  targeted  teaching  and  learning  strategies. 
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Figure  1.  A Model  for  Teaching  and  Learning  Complex  Thinking  Skills 

Authentic  Life  Situations.  Situations  of  multiple  categories  for  which  the  student  has  not  learned  answers, 
preferably  real-life  context.  Examples:  ambiguities,  challenges,  confusions,  dilemmas,  discrepancies,  doubt, 
obstacles,  paradoxes,  problems,  puzzles,  questions,  uncertainties. 

2Complex  Thinking  Skills.  Multidimensional  skills  using  more  than  one  rule  to  manage  a life  situation  or 
transforming  known  concepts  and  rules  to  fit  the  situation.  Examples:  complex  analysis,  creative  thinking,  critical 
thinking,  decision  making,  evaluation,  logical  thinking,  metacognitive  thinking,  problem  solving,  reflective  thinking, 
scientific  experimentation,  scientific  inquiry,  synthesis,  systems  analysis. 

3 Interactive  Prerequisites.  Content,  simple  thinking  skills,  and  learner  dispositions  and  habits. 

Content.  Content  includes  subject  area  content  (vocabulary,,  structure,  concept  definitions,  procedural  knowledge, 
reasoning  patterns)  and  thinking  content  (thinking  terms,  structures,  strategies,  heuristics,  and  processes). 

Simple  thinking  skills.  Simple  thinking  skills  include  cognitive  strategies,  comprehension,  concept  classification, 
discriminations,  routine  rule  using,  simple  analysis,  and  simple  application. 

Dispositions  and  habits.  Dispositions  and  habits  include  attitudes,  adaptiveness,  tolerance  for  risk,  flexibility, 
openness;  cognitive  styles  (such  as  field  dependence,  locus  of  control,  response  rates);  habits  of  mind  (persistence, 
self-monitoring,  self-reflection);  multiple  intelligences  (linguistic-verbal,  logical-mathematical,  spatial,  musical, 
bodily-kinesthetic,  interpersonal,  and  intrapersonal). 

4 Connecting  Networks  and  Operations.  Bridges  between  interactive  prerequisites  and  complex  thinking  skills 
related  to  the  life  situations  by  means  of  linkages,  schemata,  and  scaffolding. 

Linkages.  Linkages  involve  the  extension  of  prior  learning  to  the  new  context  and  higher  order  skills.  They  may 
require  mastery  or  automatization  of  prior  learning.  It  is  important  to  link  new  knowledge  and  skills  with 
prerequisites. 

Schemata.  Schemata  may  be  a network,  organization,  representation,  or  architecture  for  organizing  new  learning  so 
that  it  makes  sense  to  the  learner. 

Scaffolding.  Scaffolding  includes  the  guidance,  structure,  visual  and  verbal  representations,  and  modeling  of 
complex  thinking  skills. 

5Teaching  and  Learning  Strategies.  Teaching  and  learning  strategies  for  the  interactive  prerequisites  and  complex 
thinking  skills  provided  in  a learning  environment  that  is  considered  safe,  motivating,  and  supportive.  Learning 
progresses  in  levels,  each  elaborating  on  previous  levels,  and  connecting  previously  learned  knowledge  and  skills  to 
the  next  higher  level. 

6 Valid  Outcomes  for  Life  Situations.  Outcomes  will  be  valid  for  life  situations  when  they  are  managed  or  resolved 
within  their  context,  including  conditions,  resources,  constraints,  and  criteria. 
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Complex  thinking  skills  will  develop  more  fully  when  teachers  and  instructional  programs  align  learning  activities 
with  authentic  life  situations  and  provide  scaffolding  to  help  learners  move  from  simple  through  progressively  more 
complex  content,  processes,  and  outcomes.  In  life  situations,  individuals  must  be  able  to  select,  organize,  and 
sequence  knowledge,  thinking  skills,  and  dispositions  relevant  to  the  life  situations  that  they  encounter.  They  must 
be  able  to  apply  content  knowledge  and  thinking  models  or  strategies  to  productively  manage  various  life  situations. 

In  order  to  develop  the  cognitive  structure  that  is  most  consistent  with  complex  thinking  skills,  learners 
must  have  opportunities  to  achieve  increased  generality  and  complexity  in  thinking  applications,  with  practice 
beginning  within  the  lower-order  levels  of  content  and  learning  outcomes.  The  importance  of  developing  this 
cognitive  structure  and  the  role  of  guidance  and  practice  has  been  well  established  (Merrill,  1987).  The  benefit  for 
progressive  movement  from  simple  to  increasing  complexity  also  has  been  well  established  (Ausubel,  1968;  Clarke, 
1990;  Crowl  etal,  1 997;  Kauchak  & Eggen,  1998;  Marzano  et  al,  1988;McREL,  1997;  Wilson  & Cole,  1992). 

Authentic  Life  Situations 

Knowledge  develops  within  each  individual  through  learning  events  in  education  and  everyday  life 
(Bruner,  1990;  Dewey,  1933;  Jonassen,  1999).  Most  problem  situations  are  multicategorical  and  not  domain-specific 
(McPeck,  1990).  An  action-based  learning  environment  depends  on  immersion  with  real-world  or  authentic 
problems  (Ausubel,  1968;  Dewey,  1933;  Glaser,  1941;  Huot,  1998;  Jacobs,  1995;  Jonassen,  1999;  Jonassen,  Merrill, 
Reigeluth,  and  Rowland,  2000;  Marzano  et  al  1988;  McREL,  1997;  Vygotsky,  1978).  Yet,  life  engages  people  in 
many  situations  that  may  require  complex  thinking,  not  just  problems  to  be  solved. 

In  addition  to  problem  solving,  other  purposes  achieved  through  complex  thinking  skills  include  finding 
problems,  incompleteness,  anomalies,  troubles,  inequities,  and  contradictions;  developing  methods  of  inquiry; 
making  decisions;  making  choices,  creating  new  ideas  or  objects,  and  making  predictions  (Cotton,  1997;  Gagne, 
Briggs  & Wager,  1988;  Kahneman,  Slovic  & Tversky,  1982;  Lewis  & Smith,  1993;  McREL,  1998;  Tversky  & 
Kahneman  cited  in  Ohio  State  University,  n.d.;  Wilson  & Cole,  1992). 

Complex  Thinking  Skills 

Complex  thinking  skills  encompass  the  higher  level  skills  defined  in  Bloom's  Taxonomy  of  Educational 
Objectives  (Bloom,  1956);  the  problem-solving  level  of  skills  in  the  hierarchy  of  learning  capabilities  described  by 
Briggs  and  Wager  (1981),  Gagne  (1985),  and  Gagne,  Briggs,  and  Wager  (1988);  the  critical  thinking  skills  identified 
in  Gubbins'  Matrix  of  Critical  Thinking  Skills  (cited  in  Legg,  1990);  and  the  kinds  of  critical  thinking,  problem 
solving,  decision  making,  and  creative  thinking  skills  described  by  Lewis  and  Smith  (1993). 

Terms  used  to  describe  complex  thinking  skills  have  been  diverse,  including  active  inquiry  and  discovery, 
creative  thinking,  critical  thinking,  decision-making,  evaluation,  higher  order  thinking,  inquiry,  insight,  logical 
thinking,  metacognition,  problem  solving,  scientific  reasoning,  rational  thinking,  reflective  thinking,  synthesis,  and 
systems  analysis  (Bloom,  1956;  Bruner  cited  in  Crowl  et  al,  1997;  Cotton,  1997;  Crowl  et  al,  1997; 
Csikszentmihalyi  & Sawyer,  1995;  Dewey,  1933;  Ennis,  1989,  1993;  Facione,  1998;  Fogarty  & McTighe,  1993; 
Gagne,  Briggs  & Wager,  1988;  Gick  & Lockhart,  1995;  Glaser,  1941;  Gruber,  1995;  Haladyna,  1997;  Kahneman, 
Slovic  & Tversky,  1982;  Kauchak  & Eggen,  1998;  Legg,  1990;  Lewis  & Smith,  n.d.,  McREL,  1998;  Piaget  cited  in 
Crowl  et  al,  1997;  Pogrow,  1999;  Pogrow  & Buchanan,  1985;  Siowick-Lee,  1995;  Sternberg  & Davidson,  1995; 
Tversky  & Kahneman  cited  in  Ohio  State  University,  n.d.;  Utah  State  Office  of  Education,  1997). 

For  the  purpose  of  focusing  the  model  presented  in  this  paper,  a complex  thinking  skill  is  one  that  involves 
the  application  of  at  least  two  rules  or  principles  to  a situation  with  multiple  factors.  It  is  productive  rather  than  only 
reproductive  thinking  (Maier  cited  in  Lewis  & Smith,  n.d.,  p.  6).  It  requires  going  beyond  applying  routine  rules, 
beyond  the  routine  application  of  previously  learned  knowledge  (Newman  cited  in  Lewis  & Smith,  n.d.,  pp.  7-8).  It 
may  involve  the  putting  together  of  certain  rules  that  may  not  have  been  applied  to  previous  similar  situations 
(Gagne,  Briggs  & Wager,  1988,  pp.  65-66).  In  this  process,  concepts  and  rules  must  be  synthesized  into  new 
complex  forms  for  the  learner  to  cope  with  new  problem  situations  (Gagne,  Briggs  & Wager,  1988,  pp.  65-66). 

In  Gagne's  framework,  intellectual  skills  begin  with  discriminations  as  a prerequisite  for  concrete  and 
defined  concepts,  simple  rules,  and  then  more  complex  higher  order  rules  and  problem  solving.  The  application  of  at 
least  two  rules  defines  the  problem  solving  level  of  learning  (Briggs  & Wager,  1981;  Gagne,  1985;  Gagne,  Briggs  & 
Wager,  1988). 

Though  focused  on  process  or  biological  development  or  stages  of  development,  other  learning  theories 
also  express  levels  of  increasing  complexity.  For  example,  in  Bloom's  taxonomy,  lower  levels  of  learning  provide  a 
base  for  higher  levels  of  analysis,  synthesis,  and  evaluation.  Piaget  and  Bruner  focus  on  different  processes  for 
acquiring  skills,  but  both  include  the  importance  of  linking  previously  learned  concepts  and  information  to  new 
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learning  (Crowl  et  al,  1997;  Hummel  1997).  Bruner’s  spiral  curriculum  has  long  served  as  a model  for  developing 
higher  levels  of  complexity  over  periods  of  time  (Bruner  cited  in  Crowl  et  al,  1997).  Vygotsky  observed  that 
cognitive  development  progresses  as  children  learn  and  that  internalizing  knowledge  facilitates  higher  mental 
functions  (Crowl  et  al,  1997).  Finally  Marzano  and  McREL  have  focused  on  developing  dimensions  of  thinking  and 
learning  in  which  core  thinking  skills  and  more  complex  thinking  processes  interact  with  types  of  knowledge 
(Marzano  et  al,  1 988;  McREL  1 997). 

It  is  most  important  that  individuals  make  acquaintance  with  the  particular  facts  that  create  a need  for 
definition  and  generalization  in  order  to  see  the  correct  difficulty  to  be  overcome  not  with  definitions,  rules,  general 
principles,  classifications,  and  the  like  (Dewey,  1933,  p.  186).  Perceiving  the  correct  difficulty  is  particularly 
important  because  the  way  a problem  is  apprehended  or  defined  limits  the  kind  of  answers  that  will  occur  to  the 
thinker  (Glaser,  1941,  p.  25).  Individuals  must  be  able  to  reformulate  issues  and  steer  the  thinking  process  in  the 
right  direction. 

Individuals  must  be  able  to  combine  new  with  familiar  information  and  skills  creatively  within  limits  set  by 
the  material  or  context  (Bloom,  1956;  Bloom  cited  in  McDavitt,  1993).  The  interplay  of  multiple  intelligences, 
insight,  and  creativity  also  fits  within  the  concept  of  complex  thinking  skills. 

Intelligence,  no  longer  limited  to  the  idea  of  a single  ability  or  global  capacity  to  learn,  is  characterized  by 
multiple  dimensions  of  mental  processes,  types  of  information,  and  types  of  outcomes  involving  convergent  and 
divergent  thinking  (Crowl  et  al,  1997).  These  different  abilities  contribute  to  success  with  different  types  of  subject 
matter  content,  approaches  to  thinking  strategies,  and  ways  of  coping  with  new  and  unfamiliar  life  situations 
(Guilford  cited  in  Crowl  et  al,  1997;  Thurstone  cited  in  Crowl  et  al,  1997;  McPeck,  1990;  Gardner,  1983;  Gardner 
cited  in  Crowl  et  al,  1997;  Sternberg,  1998;  Sternberg  & Lubart,  1995;  Sternberg  cited  in  Crowl  et  al,  1997). 

Insight,  a concept  often  associated  with  creativity,  manifests  itself  in  the  sudden  unexpected  solution  to  a 
problem  (Schooler,  Fallshore,  & Fiore,  1996).  Non-insight  problems  require  routine  application  of  rules,  while 
insight  problems  require  problem  solving  and  cognitive  strategies  or  analysis,  synthesis,  and  evaluation  (Gagne, 
Briggs  & Wager,  1988;  Bloom,  1956).  Insight  may  be  a product  of  the  prepared  mind  because  only  a trained  mind 
can  make  connections  between  unrelated  events,  and  recognize  meaning  in  a serendipitous  even,  and  produce  a 
solution  which  is  both  novel  and  suitable  (Pasteur  cited  in  Crowl  et  al,  1997,  pp.  192-193).  Insight  has  been 
characterized  as  involving  the  access  of  appropriate  problem  elements,  the  search  for  a new  problem  representation, 
the  finding  of  alternative  approaches,  the  habit  of  persevering,  the  taking  of  risks,  the  application  of  broad 
knowledge,  and  the  recognition  and  use  of  analogies  (Schooler,  Fallshore  & Fiore,  1995). 

Creativity,  often  characterized  by  fluency,  originality,  and  elaboration,  requires  going  beyond  previously 
learned  concepts  and  rules  to  generate  rather  than  merely  reproduce  something  (Crowl  et  al,  1997).  Creative 
problem  solving  involves  finding  problems,  working  to  find  fresh  ways  to  view  them,  evaluating  shortcomings  and 
weaknesses,  selecting  relevant  aspects  for  attention  while  ignoring  the  irrelevant,  and  putting  the  pieces  together  in  a 
coherent  system  that  integrates  the  new  information  with  what  an  individual  already  knows  (Barron  & Harrington 
cited  in  Crowl  et  al,  1997;  Hebb,  Perkins,  & Smith  cited  in  Sternberg  & Davidson,  1995;  Sternberg  & Davidson, 
1982,  1983;  Davidson  & Sternberg  cited  in  Crowl  et  al,  1997).  Examples  of  products  resulting  from  the  creative 
process  include  Benjamin  Franklin’s  application  of  conservation  and  equilibrium  (Crowl  et  al,  1997);  Picasso's 
Guernica  resulting  from  sketches  and  modifications  of  previous  work  (Weisberg,  1995);  Coleridge's  Kubla  Khan,  a 
product  of  numerous  revisions  (Crowl  et  al,  1997);  Watson  and  Crick's  discovery  of  the  DNA  double  helix  structure 
(Weisberg,  1995;  Crowl  et  al,  1997;  Edison's  invention  of  an  electric  lighting  system  (Weisberg,  1995;  Crowl, 
1997). 

In  this  model,  metacognitive  strategies  have  been  classified  as  complex  thinking  with  the  focus  on  their 
executive  control  function— evaluating,  planning,  and  regulating  thinking  processes.  Some  metacognitive  strategies 
might  be  considered  simple  thinking  skills,  while  others  would  be  complex.  Metacognitive  strategies  include 
problem  finding  and  the  linkage  of  problem  finding  and  creativity  through  activities  of  planning,  self-monitoring  of 
progress,  and  self-adjustments  to  thinking  strategies  (Gagne,  Briggs  & Wager,  1988;  Sternberg  & Lubart,  1995; 
McREL,  1998;  Young,  1997). 

The  cognitive  development  involved  in  complex  thinking  also  leads  to  more  efficient  learning  of  both  facts 
and  skills  (DeVries  & Kohlberg,  1987;  McDavitt,  1993;  Schwartz  & Reisberg  cited  in  Crowl  et  al,  1997;  Son  & 
VanSickle,  1993). 

Interactive  Prerequisites 

Content  is  a building  block  for  thinking  skills.  The  recall  of  content  is  verbal  information.  Whether  or  not 
thinking  can  be  learned  without  content  is  only  a theoretical  point  because  education  and  life  engage  both. 
Individuals  take  current  knowledge  and  interrelate  or  rearrange  it  together  with  new  information,  using  thinking 
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skills,  which  then  function  to  extend  and  refine  knowledge  (Huot,  1998;  Marzano  et  al,  1988;  McREL,  1997).  As 
learners  build  relationships  among  concepts,  they  broaden  their  knowledge  of  the  world  and  begin  to  form  rules  to 
use  in  problem  situations  (Gagne,  Briggs  & Wager,  1988). 

Content  begins  in  relatively  simple  forms  and  grows  towards  complexity  and  the  nature  of  thinking  adapts 
to  increasing  challenge  (Clarke,  1990,  p.  24).  Mastery  of  content  and  simple  thinking  skills  are  particularly 
important  prerequisites  because  any  lesser  degree  of  learning... will  result  in  puzzlement,  delay,  inefficient  trial  and 
error  at  best,  and  in  failure,  frustration,  or  termination  of  effort  toward  further  learning  at  the  worst  (Gagne,  Briggs 
& Wager,  1988;  cf.  Bloom  cited  in  McDavitt,  1993). 

Examples  of  objectives  that  express  the  need  for  complex  thinking  skills  in  particular  subject  matter  areas 
include  the  ones  listed  below  (Florida  DOE,  1996). 

• Math:  uses  and  justifies  different  estimation  strategies  in  a real-world  problem  situation  and  determines  the 
reasonableness  of  results  of  calculations  in  a given  problem  situation. 

• Language  Arts:  selects  and  uses  strategies  to  understand  words  and  text,  and  to  make  and  confirm  inferences 
from  what  is  read,  including  interpreting  diagrams,  graphs,  and  statistical  information. 

Dispositions  and  habits  contribute  to  the  success  of  thinking  in  developing  valid  outcomes  for  life 
situations.  They  include  an  individual’s  tendencies  and  behaviors  to  seek  accuracy  and  clarity,  restrain  impulsivity, 
take  a position  or  direction,  exercise  self-regulation,  think  critically  and  creatively,  set  goals,  make  and  execute 
strategic  plans,  seek  and  evaluate  reasons  and  justifications,  analyze  and  monitor  one's  own  thinking  processes, 
sustain  intellectual  curiosity,  organizing  information  and  ideas,  persisting  when  answers  are  not  apparent,  and 
remain  open-minded  in  exploring  alternative  views  and  generating  multiple  options  (Dewey,  1933;  Fogarty  & 
McTighe,  1993;  Huot,  1998;  Marzano,  1993;  McREL,  1997). 

In  this  model,  cognitive  strategies  have  been  classified  as  simple  thinking  strategies.  They  often 
intrinsically  possess  a simple  structure  such  as  underlining  main  ideas,  outlining,  and  paraphrasing  (Gagne,  Briggs 
& Wager,  1988,  p.  70). 

Connecting  Networks  and  Operations 

Bridges  between  interactive  prerequisites  and  complex  thinking  skills  related  to  life  situations  are  formed 
by  means  of  linkages,  schemata,  and  scaffolding.  They  interweave  the  levels  of  thinking  with  content  through 
elaborating  the  given  material,  making  inferences  beyond  what  is  explicitly  presented,  building  adequate 
representations,  analyzing  and  constructing  relationships  (Lewis  & Smith,  1993,  p.  133). 

Different  processes  may  create  the  connecting  networks  and  operations  to  relate  new  conceptual  meaning 
to  previously  established  ones,  to  integrate  new  information  into  existing  schemata,  to  restructure  schemata,  or  to 
restructure  experience  (Ausubel,  1978;  Dewey,  1933;  Jonassen,  1996).  Linkages  from  the  connecting  networks  are 
critical  because  in  very  simple  terms,  we  remember  those  things  for  which  we  have  made  many  linkages  (Marzano, 
1993,  p.  156). 

Teaching  strategies  make  a difference  in  learning  outcomes  (Underbakke,  Borg  & Peterson,  1993;  Kauchak 
& Eggen,  1998;  Merrill,  Drake,  Lacy,  Pratt,  & the  ID2  Research  Group,  1996).  Methods  of  teaching  also  influence 
the  type  of  learning  outcomes.  For  example,  a Socratic  method,  or  the  use  of  stories  or  case  studies,  or  class 
discussions  may  produce  different  kinds  of  learning  (Wilson,  1997,  p.  1 1). 

Teachers  who  provide  ready-made  rules  and  generalizations  for  students  to  memorize  are  following 
practices  that  interfere  with  the  development  of  thinking  skills  (Glaser,  1941).  Novelty  and  usefulness  to  real  life, 
and  tasks  that  are  neither  too  easy  nor  too  difficult,  hold  more  promise  for  making  goals  meaningful  than  isolated, 
rote  learning  tasks.  The  synchronous  development  of  thinking  skills  with  a deepening  of  the  knowledge  base 
promotes  higher  levels  of  thinking  (Cotton,  1997;  Crowl  et  al,  1997;  Weisberg,  1995).  Cognitive  development 
through  layers  of  instruction,each  elaborating  on  the  previous  levels,  progressively  moves  the  learner  from  simple  to 
complex  concepts  and  thinking  processes  (Reigeluth,  1987;  Reigeluth  cited  in  Wilson  & Cole,  1992). 

Thinking  skills  instruction  produces  gains  on  measures  of  learning  and  intelligence  (28  studies  cited  in 
Cotton,  1997).  This  instruction  includes  multidimensional  strategies,  wait  time  after  presenting  questions  or 
problems,  and  a variety  of  examples  with  think-aloud  explanations.  It  may  include  cycles  of  analysis  and  a period  of 
incubation  or  reflection  (Crowl  et  al,  1997;  Facione,  Sanchez,  Facione  & Gainene,  1995;  Pogrow,  1990;  Pogrow  & 
Buchanan,  1985).  Practical  methods  for  infusing  thinking  into  curriculum  and  instruction  include  the  ones  listed 
below  (APA,  1997;  Dewey,  1933;  Glaser,  1941;  Huot,  1998;  Kauchak  & Eggen,  1998;  Marzano  et  al,  1988; 
McREL,  1997). 
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• Encourage  positive  attitudes,  perceptions,  and  motivation  about  learning  in  a supportive  and  safe  learning 
environment  with  respectful  and  caring  relationships. 

• Emphasize  acquisition  of  meaningful  knowledge,  especially  procedural  knowledge  as  a base  for  applied 
thinking. 

• Link,  extend,  and  deepen  knowledge  through  thinking  skills  applied  to  relevant,  authentic,  real-life  learning 
tasks. 

• Use  knowledge  and  skills  in  meaningful  authentic  tasks  over  a period  of  time. 

• Develop  dispositions,  habits  of  mind,  or  habits  of  reflection  for  organizing  information  and  thinking  and 
learning  processes,  including  creative  and  critical  thinking. 

In  addition,  it  is  important  to  provide  the  following. 

• Alignment  in  content  and  complexity  of  tasks,  assessment  activities,  and  objectives  (Kauchak  & Eggen,  1998). 

• Deliberate  design  of  activities  and  programs  to  teach  specific  thinking  and  learning  strategies  along  with  self- 
monitoring, self-reflection,  and  evaluation  (Cotton,  1997;  Darmer,  1995,  abstract;  Easterwood,  1996,  abstract; 
Glaser,  1 941 ; Kauchak  & Eggen,  1998;  Perkins  & Salomon,  1989). 

• Directions  on  use  of  cognitive  strategies  such  as  methods  of  rehearsal,  elaboration,  organization,  reflection,  and 
paraphrasing  to  improve  one's  own  learning  (Cotton,  1997;  Crowl  et  al,  1997). 

• Novel  problems  and  questions  to  evoke  thinking  (Dewey,  1933;  Kauchak  & Eggen,  1998). 

• Emphasis  of  broad  problem  solving  strategies,  algorithms,  or  heuristics  (Crowl  et  al,  1997;  Kauchak  & Eggen, 
1998). 

• Clarity  of  instructions  and  assignments,  e.g.,  explain  the  nature  of  the  thinking  task  (Kauchak  & Eggen,  1998). 

• Organized  activities  and  structure  for  processes  (Kauchak  & Eggen,  1998). 

• Choices  among  assignments  (Kauchak  & Eggen,  1998;  Crowl  et  al,  1997). 

• Emphasis  on  transfer  of  skills  to  everyday  life  situations  by  including  conditions  of  real-life  in  practice 
opportunities  (Kasonen  & Winne,  1995;  Perkins  & Salomon,  1989;. 

• Scaffolding  of  just  enough  support  to  guide  students  until  they  can  perform  skills  independently  (organizing 
frameworks,  hints,  questions,  examples  with  explanations,  modeling,  corrective  and  specific  feedback  ) 
(Kauchak  & Eggen,  1998;  Slavin,  1995;  Vygotsky  in  Crowl  et  al,  1997;  McREL,  1997). 

• Explanation  and  modeling  of  habits  of  thinking  and  dispositions  such  as  persistence  (Crowl  et  al,  1997; 
Kauchak  & Eggen,  1998). 

• Activities  that  use  various  types  of  intelligences  and  encourage  intellectual  diversity  (Gardner  in  Crowl,  1997; 
Gardner  cited  in  Kauchak  & Eggen,  1998;Kauchak  & Eggen,  1998;  Merrill,  2000). 

• Open-ended  tasks  involving  several  ways  to  resolve  difficulties  or  solve  problems  and  that  give  opportunities 
for  small  groups  to  contribute  to  outcomes  (Kauchak  & Eggen,  1998). 

• Visual  and  verbal  representations  and  explanations  (drawings,  graphs,  maps,  tales,  hierarchies,  lists  of  steps) 
(Clarke,  1990;  Crowl  etal,  1997;  Glaser,  1941 ; Kauchak  & Eggen,  1998). 

• Practice  in  making  inferences,  in  deciding  how  and  when  to  apply  different  types  of  thinking  skills,  and  in 
producing  outcomes  for  a variety  of  life  situations  (Crowl  et  al,  1997;  Howe  & Warren,  1989;  Kauchak  & 
Eggen,  1998). 

• Frequent,  short  assignments  to  include  learning  of  prerequisite  knowledge  and  skills  and  drill  and  practice 
using  verbal  analogies,  logical  reasoning,  inductive  and  deductive  thinking,  and  discrete  steps  and  linkages 
involved  in  complex  thinking  processes  (Kauchak  & Eggen,  1998). 

• Varieties  of  structured  peer  tutoring,  cooperative  learning,  collaborative  small  group  work,  team  assisted 
individualization  with  individual  responsibilities  and  group  products  (Kauchak  & Eggen,  1998;  Kewley,  1996, 
abstract).  [Social  interaction  is  one  of  the  vehicles  by  which  learners  share  information  (Vygotsky  cited  in 
Crowl  et  al,  1997)]. 

• Small  group  discussions  only  after  assuring  presence  of  prerequisite  content  knowledge  and  thinking  skills 
(Kauchak  & Eggen,  1998). 

• Limited  direct  teaching,  and  if  used,  focus  on  mini-lectures  combined  with  activities  such  as  guided  practice, 
demonstrations,  debates,  student  questions,  reviews  and  summaries  (Patrick,  1986;  Crowl  et  al,  1997;  Kauchak 
& Eggen,  1998). 

• Questioning  strategies  to  stimulate  curiosity  of  all  learners,  beginning  with  lower-order  questions  and 
progressively  leading  up  to  more  complex  questions  (Crowl  et  al,  1997;  Kauchak  & Eggen,  1998). 

• Monitoring,  feedback,  redirection,  and  correction  of  inefficient  or  incorrect  strategies  and  pursuit  of  dead-end 
or  simplistic  answers  (Cotton,  1997;  Crowl  et  al,  1997). 
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• Feedback  in  the  form  of  informal  checks,  immediate,  with  positive  emotional  tones,  specific  and  non 
judgmental,  simple  correction  of  errors  without  overexplaining  (Crowl  et  al,  1997;  Kauchak  & Eggen,  1998). 

• Reinforcement  and  encouragement  of  targeted  thinking  skills  and  progress  of  learners  to  develop  their 
confidence  and  a greater  sense  of  locus  of  control  (Cotton,  1997;  Crowl,  1997). 

• Adaptations  for  diverse  learner  needs  (Kauchak  & Eggen,  1998). 

• Mastery  skills  development  and  test  management  for  subskills  and  prerequisite  knowledge  (Kauchak  & Eggen, 
1998). 
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Abstract 

Social  desirability  responding  (SDR)  on  surveys  administered  on  the  World  Wide  Web  (WWW)  and  on 
paper  was  examined  using  178  graduate  and  undergraduate  students  randomly  assigned  to  a 2 (survey's 
administration  mode:  WWW  and  paper)  X 2 (participants'  identifiability  level:  anonymous  and  non-anonymous)  true 
experimental  design.  The  findings  reveal  no  differences  in  SDR  between  the  WWW  and  the  paper-administered 
survey  conditions,  and  no  differences  in  SDR  between  the  anonymous  and  non-anonymous  conditions.  These 
findings  and  potential  explanations  are  examined  for  consideration  by  anyone  interested  in  using  the  WWW  to 
obtain  accurate  information  from  survey  participants. 

Introduction 

Response  bias  (i.e.,  the  systematic  tendency  to  respond  to  surveys,  questionnaires,  standardized  tests,  and 
other  self-report  measures  on  some  basis  other  than  the  specific  item  content)  continues  to  confound  research 
findings.  People's  reports  of  their  own  traits,  attitudes,  and  behaviors  often  involve  systematic  bias  that  obscures 
measurement  of  content  variables  (Calsyn,  1999;  Paulhus,  1991).  For  example,  early  research  suggested  that 
standard  self-report  methodologies  distorted  the  reporting  of  racist  attitudes  (Sigall  & Page,  1971),  abnormal  sexual 
attitudes  (Knudson,  Pope,  & Irish,  1967),  desirable  behaviors  (Phillips  & Clancey,  1972),  deviant  behaviors  (Clark 
& Tifft,  1966),  and  abortion  (Wiseman,  1972).  More  recent  studies  have  revealed  a tendency  among  individuals  to 
conceal  truth  when  reporting  unverifiable  information  (Lautenschlager  & Flaherty,  1990),  seeking  employment 
(Calsyn  & Klinkenberg,  1995),  reporting  information  designed  to  impress  others  (Rosenfeld,  Giacalone,  & Riordan, 
1995),  and  when  a respondent's  anonymity  is  violated  (Sproull  & Kiesler,  1991). 

Among  the  most  common  forms  of  response  bias  reported  in  the  literature  are  deviant  responding  (Berg, 
1967),  careless  responding  (Meehl  & Hathaway,  1946),  consistent  responding  (Dillehay  & Jemigan,  1970),  item 
omission  (Cronbach,  1946),  acquiescence  (Ray,  1983),  and  extremity  bias  (Hamilton,  1968;  Peabody,  1962). 
However,  perhaps  the  most  frequently  studied  response  bias  is  social  desirability  responding  (SDR)  (i.e.,  the 
tendency  to  provide  answers  which  cause  the  respondent  to  look  good)  (Rosenfeld,  Booth-Kewley,  Edwards,  & 
Thomas,  1996).  As  early  as  the  1930s,  Bernreuter  (1933)  reported  that  psychometricians  had  already  noted  the 
problem  of  SDR  effects  on  the  validity  of  questionnaires.  Years  later,  Meehl  and  Hathaway  (1946)  were  able  to  cite 
eight  measures  specifically  designed  to  index  SDR  in  self-report  measures.  Since  that  time,  SDR  has  been  a major 
concern  in  measuring  personality,  psychopathology,  attitudes,  and  self-reports  of  various  forms  of  sensitive  behavior 
(Paulhus,  1991). 

Recently,  the  proliferation  of  web-based  and  other  computer-assisted  means  of  acquiring  information  from 
individuals  has  raised  concerns  regarding  how  responses  obtained  through  computers  compare  with  responses 
obtained  on  paper  instruments.  Several  published  studies  (see  Booth-Kewley,  Rosenfeld,  & Edwards,  1993; 
Moorman  & Podsakoff,  1992,  for  reviews)  have  reported  that  computer  responses  are  more  candid,  less  biased,  and 
less  influenced  by  social  desirability  than  responses  given  on  paper.  However,  very  few  studies  have  examined 
people’s  responses  on  the  World  Wide  Web  (WWW).  Furthermore,  several  research  efforts  have  failed  to  replicate 
the  findings  of  previous  studies  regarding  SDR.  For  example,  studies  in  the  1980s  investigating  the  feasibility  of  a 
U.S.  Navy  computer-based  survey  system  failed  to  demonstrate  that  computer-administered  surveys  were  superior  at 
reducing  SDR  (Doherty  & Thomas,  1986;  Rosenfeld,  Doherty,  Vicino,  Kantor,  & Greaves,  1989;  Vicino,  1989).  In 
addition,  Lautenschlager  and  Flaherty  (1990)  found  that  undergraduates  responding  to  a computerized  survey  had 
higher  scores  on  a self-deception  scale  than  did  their  counterparts  who  used  a paper  survey.  Extending  this  research, 
Booth-Kewley  et  al.  (1993)  had  subjects  complete  surveys  using  computer  or  paper  administration  modes  under 
anonymous  or  identified  conditions.  Consistent  with  Lautenschlager  and  Flaherty's  (1990)  results,  Booth-Kewley 
and  her  colleagues  found  that  identified  respondents  gave  more  socially  desirable  responses  than  did  their 


173 


182 


anonymous  counterparts.  However,  Booth-Kewley  et  al.  (1993)  failed  to  replicate  Lautenschlager  and  Flaherty's 
(1990)  finding  of  greater  levels  of  SDR  among  participants  who  completed  the  survey  by  computer. 

The  lack  of  consistent  findings  regarding  the  extent  to  which  people  demonstrate  SDR  on  computer  versus 
paper-administered  instruments,  combined  with  the  scarcity  of  research  in  this  area  on  people  who  respond  to 
surveys  and  questionnaires  administered  through  the  WWW,  was  the  purpose  for  conducting  the  current  study. 
Based  on  previous  research  suggesting  that  computer-administered  surveys  yield  more  candid  responses  than  do 
paper  surveys,  this  study  hypothesized  that  adult  students  taking  a survey  on  the  WWW  would  demonstrate 
significantly  less  SDR  than  would  students  taking  the  same  survey  on  paper.  Furthermore,  based  on  previous 
research  suggesting  that  participants  would  be  more  inclined  to  respond  to  survey  items  under  conditions  of 
anonymity,  this  study  hypothesized  that  adult  students  taking  the  survey  anonymously  would  demonstrate 
significantly  less  SDR  than  would  respondents  who  were  asked  to  identify  themselves. 

The  Study 

Participants 

178  undergraduate  and  graduate  students  at  a large  university  in  the  southeastern  United  States,  enrolled  in 
introductory  research  and  technology  courses,  participated  in  this  study.  69%  of  the  participants  were  female.  The 
average  age  of  participants  was  34.2  years. 

Instrument 

To  assess  the  extent  to  which  participants  would  demonstrate  SDR,  this  study  used  the  Balanced  Inventory 
of  Desirable  Responding  (BIDR)  (Paulhus,  1993).  The  BIDR  consists  of  40  items  stated  as  propositions. 
Respondents  rate  their  agreement  with  each  statement  on  a seven-point  scale.  The  scoring  key  is  balanced.  After 
reversing  the  negatively  keyed  items,  one  point  is  added  for  each  extreme  response  (six  or  seven).  This  method  of 
scoring  ensures  that  high  scores  arc  attained  only  by  subjects  who  give  exaggeratedly  desirable  responses.  The  40 
items  are  then  summed  to  yield  an  individual's  overall  level  of  SDR. 

Several  studies  have  established  the  reliability  and  validity  of  the  BIDR.  With  respect  to  internal  validity, 
values  of  coefficient  alpha  have  ranged  from  .68  to  .80  and  from  .75  to  .86  (Mellor,  Conroy,  & Masteller,  1986; 
Paulhus,  1984,  1993;  Quinn,  1989).  Demonstrating  concurrent  validity  as  a measure  of  SDR,  the  sum  of  the  40 
items  on  the  BIDR  correlated  .71  with  the  Marlowe-Crowne  Social  Desirability  scale  (Crowne  & Marlowe,  1960) 
and  .80  with  the  Multidimensional  Social  Desirability  Inventory  of  Jacobson,  Kellogg,  Cauce,  and  Slavin  (1977). 
Supporting  the  construct  validity  of  the  BIDR,  Paulhus  (1991)  discovered  that  high  deception  subjects  were  more 
likely  than  lows  to  show  a self-serving  bias  after  a failure  experience.  High  self-deception  subjects  also  showed 
more  illusion  of  control,  belief  that  they  were  safe  drivers,  and  proneness  to  love  (Paulhus  & Reid,  1991)  and  to 
intrinsic  religiosity  (Leak  & Fish,  1989). 

Procedures 

Using  a true  experimental  design,  this  study  examined  the  impact  of  two  independent  variables  — the 
participants'  identifiability  level  (i.c.,  anonymous  and  non-anonymous)  and  the  survey's  administration  mode  (i.e., 
WWW-administered  and  paper-administered)  — on  one  dependent  variable  — the  participants’  social  desirability 
response  levels  measured  by  the  BIDR.  Using  a random  number  table,  283  potential  participants  were  assigned 
randomly  to  one  of  four  conditions:  (a)  anonymous/WWW-administered  survey  (n=75);  (b)  non- 

anonymous/WWW-administered  survey  (n-78);  (c)  anonymous/paper-administered  survey  (n=63);  and  (d)  non- 
anonymous/paper-administered  survey  (n=67). 

Prior  to  their  departure  from  the  classroom  during  an  initial  session  in  the  course,  participants  in  all  four 
conditions  were  provided  a manila  envelope  by  their  professor,  who  was  instructed  to  say,  “Prior  to  our  next  class 
attendance,  please  follow  the  instructions  contained  in  your  envelope . Those  instructions  will  require  you  to 
individually  complete  a vefy  short  survey  that  we  will  use  later  in  this  course  when  we  discuss  data  collection 
techniques.  Remember,  please  follow  the  instructions  contained  in  your  envelope  by  our  next  class  attendance. 
Thank  you.  ” 

For  the  participants  in  the  paper-administered  survey  groups,  the  manila  envelopes  contained  a copy  of  the 
Balanced  Inventory  of  Desirable  Responding  (BIDR).  One-half  of  the  BIDRs  in  those  envelopes  (i.e.,  the  non- 
anonymous/paper-administered  survey  group)  were  preceded  by  the  written  instructions,  " Please  complete  the 
following  survey  and  mail  it  in  the  self-addressed  envelope  prior  to  our  next  class  attendance.  The  survey  contains 
40  items  and  takes  approximately  10  minutes  to  complete . Using  the  scale  below  as  a guide,  write  a number  in  each 
blank  to  indicate  your  agreement  with  the  statement.  For  the  purpose  of  accountability,  please  be  certain  that  you 


174 


183 


print  your  name  legibly  in  the  blank  provided.  The  results  of  this  surx'ey  will  be  completely  confidential.  The  residts 
will  be  reported  only  in  the  aggregate;  therefore,  at  no  time  will  you  be  identified  individually." 

The  other  one-half  of  the  BTDRs  in  those  envelopes  (i.e.,  the  anonymous/paper-administered  survey  group) 
were  preceded  by  the  same  written  instructions,  with  one  exception  — the  phrase,  " For  the  purpose  of 
accountability,  please  be  certain  that  you  print  your  name  legibly  in  the  blank  provided , " was  replaced  with  the 
phrase,  “7b  maintain  anonymity , please  do  not  indicate  your  name  anywhere  on  the  survey.  " 

For  the  participants  in  the  WWW-administcrcd  survey  groups,  onc-half  of  the  manila  envelopes  (i.e.,  the 
non-anonymous/WWW-administered  survey  group)  contained  a sheet  of  paper  with  the  written  instructions,  "Please 
complete  the  survey  that  you  will  find  on  the  World  Wide  Web  at  http://education.uncc.edu/surveyl.htm  prior  to  the 
next  class  attendance.  The  results  of  this  survey  will  be  completely  confidential.  The  results  will  be  reported  only  in 
the  aggregate;  therefore,  at  no  time  will  you  be  identified  individually."  For  participants  in  this  condition,  the  BIDR 
at  the  prescribed  web  site  address  included  the  following  additional  instructions,  "This  survey  contains  40  items  and 
takes  approximately  10  minutes  to  complete.  Using  the  scale  below  as  a guide,  type  a number  in  each  blank  to 
indicate  your  agreement  with  the  statement.  For  the  purpose  of  accountability,  please  be  certain  that  you  type  your 
name  in  the  blank  provided. " 

The  other  onc-half  of  these  manila  envelopes  (i.e.,  the  anonymousAVWW-administered  survey  group) 
contained  a sheet  of  paper  with  the  same  written  instructions,  with  two  exceptions  — the  WWW  address  for 
accessing  the  survey  for  participants  in  this  condition  was  changed  from  “ education. uncc.edu/survev l .htm  ” to 
" education,  uncc.edu/  respond  1 .htrn,  ” and  the  phrase,  ‘Tor  the  purpose  of  accountability,  please  be  certain  that  you 
type  your  name  in  the  blank  provided , " was  replaced  with  the  phrase,  ‘To  maintain  anonymity,  please  do  not 
indicate  your  name  anywhere  on  the  survey. " 

Of  the  283  potential  participants  in  this  study,  181  students  completed  a survey  (i.e.,  a 64%  response  rate). 
However,  three  students’  survey  responses  were  excluded  from  the  data  analysis  because  these  students  incorrectly 
completed  the  survey.  Of  the  remaining  178  surveys,  44  surveys  were  in  the  anonymous/World  Wide  Web- 
administered  condition  (i.e.,  a 59%  response  rate),  50  surveys  were  in  the  non-anonymous/World  Wide  Web- 
administered  condition  (i.e.,  a 64%  response  rate),  44  surveys  were  in  the  anonymous/paper-administered  condition 
(i.e.,  a 70%  response  rate),  and  40  surveys  were  in  the  non-anonymous/paper-administered  condition  (i.e.,  a 60% 
response  rate). 

A 2x2  analysis  of  variance  (ANOVA)  was  conducted  with  the  survey’s  administration  mode  (i.e.,  WWW- 
administered  and  paper-administered)  and  the  participants’  identifiability  level  (i.e.,  anonymous  and  non- 
anonymous)  as  the  independent  variables  and  the  participants’  SDR  levels  measured  by  the  BIDR  as  the  dependent 
variable. 

Findings 

Means,  standard  deviations,  and  sample  sizes  for  the  social  desirability  levels  by  condition  are  presented  in 
Table  1.  The  results  of  the  ANOVA  are  presented  in  Table  2.  The  main  effect  for  survey  administration  mode  was 
not  statistically  significant  (/^  1,1 74)=.  07 1,  p>. 05).  Students  taking  the  survey  on  the  WWW  (A/=  15.33,  sd=6.23) 
did  not  demonstrate  significantly  less  SDR  than  did  adult  students  taking  the  same  survey  on  paper  (A/=15.07, 
^=5.50).  Furthermore,  the  main  effect  for  participants'  identifiability  level  was  not  statistically  significant 
(F(  1,1 74)=.  150,  p>. 05).  Students  taking  the  survey  anonymously  (M=15.03,  sd-5.41)  did  not  demonstrate 
significantly  less  SDR  than  did  survey-takers  who  were  asked  to  identify  themselves  (Af=15.39,  sd=6. 29).  Finally, 
there  was  no  statistically  significant  interaction  (T(l,174)=.027,  p>. 05),  suggesting  that  no  differential  effect  on 
SDR  was  noted  with  the  combination  of  independent  variables. 

Conclusions 

The  hypotheses  of  this  study  were  based  on  previous  research;  therefore,  it  is  important  to  examine  possible 
reasons  for  the  non-statistically  significant  findings.  One  explanation  may  be  that,  unlike  the  current  study,  most 
previous  studies  linking  lower  anonymity  with  higher  levels  of  SDR  administered  their  surveys  within  the  context  of 
the  experimental  setting.  For  example,  Lautenschlager  and  Flaherty  (1990)  asked  college  students  to  complete  the 
BIDR  in  a small  office  on  the  site  of  the  study.  Similarly,  Rosenfeld,  Booth-Kewley,  Edwards,  and  Thomas  (1996) 
administered  the  BIDR  to  Navy  recruits  in  a large  testing  room  immediately  after  the  sailors  received  instructions 
about  the  study.  However,  in  the  current  study,  after  the  professor  distributed  the  manila  folders,  students  were 
allowed  to  depart  the  classroom  with  the  expectation  that  they  would  complete  the  survey  on  the  WWW  or  paper 
prior  to  the  next  lesson.  Undoubtedly,  students  completed  the  survey  in  many  different  locations  — at  home,  at 
work,  at  school,  in  a computer  lab,  and  so  forth.  As  a result,  even  those  students  in  the  non-anonymous  WWW  and 
paper-administered  survey  conditions  whose  instructions  included  a directive  to  type  or  print  their  names  on  the 
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survey  may  have  felt  a sense  of  anonymity  as  they  completed  the  BIDR.  This  pervasive  sense  of  anonymity 
experienced  by  the  participants  may  have  mitigated  the  effects  of  self-identification  created  in  the  study,  thereby 
contributing  to  the  lack  of  a statistically  significant  main  effect  for  participants'  identifiability  level. 

Another  potential  contributor  to  this  study's  findings  may  have  been  the  written  instructions  provided  to  the 
participants.  In  all  experimental  conditions,  the  written  instructions  included  the  sentences,  " The  results  of  this 
survey  will  be  completely  confidential.  The  results  will  be  reported  only  in  the  aggregate;  therefore , at  no  time  will 
you  be  identified  individually . " Although  these  sentences  were  inserted  to  ensure  compliance  with  the  1974  Buckley 
Amendment,  their  inclusion  may  have  lessened  the  extent  to  which  participants  in  the  non-anonymous  conditions 
perceived  that  they  could  be  identified.  In  effect,  these  sentences  may  have  caused  all  participants  to  believe  that 
their  identity  was  completely  protected,  thereby  mollifying  the  impact  of  anonymity  on  participants*  demonstration 
of  SDR. 

Furthermore,  research  has  suggested  that  a survey-taker's  perception  of  the  verifiability  of  her  or  his  survey 
responses  may  impact  the  extent  to  which  the  survey-taker  stretches  the  truth  in  an  effort  to  make  a good  impression 
(Lautenschlager  & Flaherty,  1990).  Specifically,  when  respondents  believe  that  their  answers  cannot  be  validated, 
they  tend  to  exhibit  higher  levels  of  SDR  than  when  they  think  that  their  responses  are  verifiable.  In  the  current 
study,  however,  participants  were  told  that  the  results  of  the  survey  would  be  used  in  an  upcoming  discussion  of  data 
collection  techniques.  As  a result,  participants  may  have  believed  that  the  survey  responses  were  being  verified, 
thereby  negating  any  SDR  effects  prompted  by  the  method  of  survey  administration  (i.e.,  WWW  or  paper). 

In  the  past,  one  factor  often  associated  with  lower  levels  of  SDR  in  computer-administered  survey 
responses  than  in  paper-administered  survey  responses  has  been  the  standardization  that  computer  administration 
affords  (Feuer,  1986).  In  computer-administered  surveys,  SDR  may  be  reduced  by  controlling  the  respondent's 
ability  to  preview,  skip,  review  items,  and  change  responses.  In  other  words,  the  greater  structure  imposed  by  the 
computer  mode  of  survey  administration  may  limit  respondents’  ability  to  reveal  themselves  in  the  best  possible 
light.  However,  in  the  current  study,  the  survey's  presentation  on  the  WWW  was  designed  to  maximize  participants' 
freedom  to  negotiate  the  instrument.  Students  who  accessed  the  BIDR  through  the  WWW  had  cpmplete  latitude  to 
preview,  skip,  change,  and  review  their  responses  to  the  items  prior  to  submitting  their  surveys  electronically.  As  a 
result,  the  restrictions  often  inherent  in  computer  survey  administration  which  lead  to  lower  levels  of  SDR  were  not 
evident  in  this  study,  perhaps  contributing  to  the  non-statistically  significant  main  effect  for  survey  administration 
mode. 

Finally,  although  early  research  revealed  that  computer-administered  survey  settings  seemed  to  reduce  SDR 
because  those  settings  offered  greater  anonymity  and  were  perceived  as  impersonal  and  nonjudgmental,  recent 
studies  have  discovered  a growing  concern  among  many  survey-takers  that  computers  are  becoming  overly  intrusive 
(Rosenfeld,  Booth-Kewley,  Edwards,  & Thomas,  1996).  This  concern,  sometimes  labeled  the  'big  brother 
syndrome"  (Martin  & Nagao,  1989),  suggests  that  people  are  becoming  more  aware  that  computer  communications 
can  be  monitored  and  shared.  Computer-users  who  suspect  "big  brother  monitoring"  have  reported  increased 
anxiety,  fatigue,  stress,  and  reduced  job  satisfaction  (Eisman,  1991;  Iadipaolo,  1992).  In  the  current  study,  students 
in  the  computer-administered  survey  condition  may  have  felt  that  their  responses,  even  in  the  anonymous  condition, 
could  and  perhaps  would  be  traced  to  them  through  the  WWW.  As  a result,  participants  may  have  felt  less  inclined 
to  present  themselves  in  a truthful  manner. 

The  growing  popularity  of  computers  throughout  much  of  the  world  suggests  that  computer  administration 
of  surveys  will  continue  to  increase  in  the  future.  Therefore,  it  is  important  to  know  how  survey  responses  obtained 
through  computers  compare  with  responses  obtained  on  paper  instruments.  Although  some  research  has  reported 
that  computer  responses  are  more  candid,  less  biased,  and  less  influenced  by  social  desirability  than  responses 
provided  on  paper,  the  current  study  using  the  WWW  as  the  means  by  which  to  administer  computer-based  surveys 
did  not  support  these  findings.  Students  taking  a survey  on  the  WWW  did  not  demonstrate  significantly  less  SDR 
than  did  students  taking  the  same  survey  on  paper,  and  students  taking  a survey  anonymously  did  not  demonstrate 
significantly  less  SDR  than  did  survey-takers  who  are  asked  to  identify  themselves.  Unlike  traditional  computer- 
based  modes  of  survey  administration  in  which  people  demonstrate  less  SDR  than  people  taking  a survey  by  paper, 
this  study  suggests  that  people  who  take  surveys  on  the  WWW  demonstrate  SDR  at  levels  comparable  to  those  who 
take  paper  surveys.  Although  increased  objectivity  and  cost  effectiveness  have  often  been  associated  with  computer 
administration  of  surveys  relative  to  paper  administration  of  surveys,  this  study's  findings  should  be  considered 
carefully  by  all  professions  interested  in  using  the  WWW  to  obtain  truthful  and  accurate  information  from  survey- 
takers. 
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Table  1 


Means , Standard  Deviations , and  Sample  Sizes  of  Social  Desirability  Responding  Level  by  Participants' 
Identifiability  Level  and  Survey's  Administration  Mode 


Administration  Mode 


World  Wide  Web-  Paper- 

Administered  Administered  Overall 


M 

SD 

n 

M 

SD 

n 

M 

SD 

n 

Participants'  Identifiability 

Anonymous 

15.07 

5.73 

44 

14.98 

5.25 

44 

15.03 

5.47 

88 

Non-anonymous 

15.56 

6.69 

50 

15.18 

5.83 

40 

15.39 

6.29 

90 

Overall 

15.33 

6.23 

94 

15.07 

5.50 

84 

15.21 

5.88 
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Table  2 

2x2  ANOVA  of  Effects  of  Participants'  Identifiability  Level  and  Survey's  Administration  Mode  on  Social  Desirability 
Responding  Level 

SS 

df 

MS 

F 

S.g 

Participants'  Identifiability  Level  (PIL) 

5.256 

1 

5.256 

.150 

.699 

Survey's  Administration  Mode  (SAM) 

2.504 

1 

2.504 

.071 

.790 

PIL  x SAM 

.956 

1 

.956 

.027 

.869 

Residual 

6115.868 

174 

35.149 

Note:  £ > .05 
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Abstract 

The  use  of  Web-based  and  Web-enhanced  instruction  is  growing  rapidly.  There  is  much  information  and 
discussion  surrounding  the  layout  of  content  on  the  screen  and  facilitating  communication  yet  little  information  is 
available  regarding  the  students  perception  and  use  of  online  instruction.  Everyone  knows  that  students  print 
information  available  on  the  Web  but  not  why  nor  how  this  affects  their  satisfaction  with  the  course.  This  study  was 
designed  to  identify  factors  that  influence  students  to  print  course  information  delivered  online,  to  ascertain  if  these 
printing  activities  are  related  to  reducing  barriers,  and  to  determine  if  these  printing  activities  influence  students’ 
satisfaction  for  learning  via  the  Web.  Implications  for  the  design  and  development  of  Web-based  instruction  are 
discussed. 

Designers  and  developers  of  Wcb-based  instruction,  specifically  Web-enhanced  instruction,  assume  that 
students  use  the  information  in  the  format  it  was  designed  that  is,  online.  If  fact,  many  learners  actually  transfer  this 
information  to  paper  by  printing  it.  This  study  was  designed  to  identify  factors  that  influence  students  to  print 
course  information  delivered  online,  to  ascertain  if  these  printing  activities  were  related  to  reducing  barriers,  and  to 
determine  if  these  printing  activities  influence  students’  satisfaction  for  learning  via  the  Web.  These  possibilities 
have  implications  for  the  design  of  Web-based  instruction. 

As  interest  in  Web-based  instruction  (WBI)  has  grown,  much  of  the  discussion  in  professional  circles  has 
revolved  around  the  mechanics  of  which  software  to  use  (McCollum,  1997);  how  and  when  to  incorporate  visuals 
(Bixler  & Spotts,  1998;  Dana,  1998;  Milheim  & Rezabek,  1997);  and  how  to  manage  e-mail,  listserv  and  threaded 
discussions  (Friedlander  & Kerns,  1998).  In  addition  to  spending  time  on  the  mechanics  of  Web  page  development, 
a considerable  amount  of  the  WBI  designer's  time  is  devoted  to  the  design  and  layout  of  the  screen  or  interface 
(Bixler  & Spotts,  1999;  Carr  & Peters,  1998;  Horton  & Lynch,  1998;  Ipek,  1999;  Kahn,  1999;  McCormack  & Jones, 
1998;  Montgomery,  1998).  Finally,  current  research  has  focused  on  online  communication  issues  such  as  email, 
bulletin  boards,  and  threaded  discussions  (Jiang,  1998;  McCabe,  1997;  Nasseh,  1996).  These  discussions  do  not 
address  the  aspects  of  evaluating  learning  opportunities  via  the  Web,  nor  do  they  address  concerns  arising  when 
students  transfer  information  from  the  computer  screen  to  the  printed  page. 

The  systematic  approach  to  the  design  of  instruction  (Dick  & Carey,  1990;  Gagne,  Briggs  & Wager,  1992; 
Reigeluth,  1999;  Seels  & Glasgow,  1997;  Smith  & Ragan,  1993)  suggests  that  the  analysis  of  student  characteristics 
is  critical  to  the  design  of  effective  instruction.  There  is  little  research  available  which  tells  the  instructional 
designer,  course  developer,  or  instructor,  how  students  are  actually  using  online  course  materials.  Ward  (1998) 
found  that  many  articles  have  been  published  on  instructional  design  issues,  technical  considerations,  and 
accreditation  issues  of  distance  education  courses  but  little  has  been  published  regarding  the  online  distance  learner's 
experience  in  his  or  her  own  words.  Determining  the  form  in  which  students  prefer  to  use  information,  and  what 
their  learning  needs  are  is  a crucial  design  component  for  designers  of  WBI. 

Instructional  Design  and  the  Web-based  Instructor 

The  design  of  effective  instructional  materials  is  a complex  task.  The  designer  asks  what  is  the  message 
being  delivered  to  the  learner  and  what  is  the  best  way  to  deliver  that  message.  With  the  advent  of  Web-based 
instruction,  an  entire  new  set  of  concerns  is  presented  to  the  instructional  developer.  These  include  questions  such 
as  how  to  break  the  information  up  into  usable  pieces  and  how  to  guide  the  student  through  the  now  nonlinear 
information.  At  first,  screen  layout  and  design  were  based  on  well-established  theories  of  layout  for  the  printed 
page. 

From  the  beginning  of  their  educational  experience,  students  learn  to  learn  from  the  printed  page. 
Researchers  have  had  years  of  opportunity  to  study  this  method  of  information  delivery.  As  Houle  (1996)  notes  for 
print  based  information,  much  is  known  about  such  matters  as  readability,  the  display  of  exhibited  material,  the  way 
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potential  for  further  follow-up  as  there  was  no  way  to  determine  which  students  in  a class  had  not  responded  and  to 
then  contact  them  individually. 

As  responses  were  submitted  by  the  participants  the  multiple-choice  questions  were  tallied  automatically  by 
the  WebCT  program.  The  researcher  compiled  the  short  answer  and  paragraph  responses  for  further  analysis.  The 
qualitative  questions  were  coded  based  on  similar  and  related  themes  of  response. 

Underlying  Assumption 

The  assumption  was  made  that  students  do  print  the  information  made  available  on  a course  Web  site.  One 
hundred  seventy- five  participants  (85.37%)  reported  they  printed  pages  from  the  course  Web  site.  This  number 
substantiates  the  assumption.  Eighty-two  participants  (40%)  reported  they  printed  50  or  more  pages  from  the  course 
Web  site.  The  average  number  of  pages  printed  was  approximately  30  (mean  = 4.79,  SD  = 2.30). 

Factors  that  Influence  Students  to  Print 

The  primary  research  question  determined  factors  that  influenced  students  to  print  online  course 
information. 

Computer  Access 

Requiring  students  to  access  a computer  to  obtain  course  materials  or  participate  in  discussion  forced  them 
to  be  in  a particular  location,  at  the  computer.  The  majority  of  participants  used  a computer  at  home  (n  = 148, 
72.20%)  or  a computer  in  a lab  on  campus  (n  = 143,  69.76%).  Twenty-two  of  the  participants  (10.73%)  indicated 
they  used  a computer  at  work  and  42  of  the  participants  (20.49%)  indicated  they  used  a computer  in  other  locations. 
Eighty  of  the  participants  (39.24%)  used  a computer  both  in  a lab  on  campus  and  at  home.  It  is  important  to  note 
that  analysis  of  the  data  in  this  study  indicated  that  the  actual  location  of  the  computer,  for  example  at  home  or  in  a 
computer  lab,  did  not  affect  student  satisfaction  with  the  course,  students’  perception  that  printing  affected 
satisfaction  with  the  course,  or  the  number  of  pages  printed. 

Reasons  for  not  Printing 

The  primary  reason  for  not  printing,  reported  by  12  participants  (5.80%),  was  to  save  paper.  The  next  most 
frequently  reported  reason,  reported  by  8 participants  (3.90%),  was  that  there  were  too  many  pages  to  print  and  it 
took  too  long.  Although  19  participants  who  did  not  print  provided  reasons  related  to  cost,  (too  many  pages,  to  save 
paper,  and  don't  own  a printer)  cost  did  not  appear  to  be  a major  factor  in  this  study. 

Reasons  for  Printing 

The  data  indicate  that  course  information  on  the  Web  was  not  accessible  during  class,  was  not 
transportable,  was  not  easy  to  reference,  and  was  not  tangible.  The  most  common  reason  for  printing,  (n=71, 
40.57%)  was  that  participants  printed  the  information  to  study.  This  was  followed  by;  to  be  able  to  take  the 
information  with  them  to  other  locations  (n=46,  26.29%),  for  easy  reference  to  the  materials  (n=43,  24.57%),  to  be 
able  to  take  notes  on  the  pages,  highlight  them,  and  use  them  during  lecture  (n=36,  20.56%),  and  it  is  hard  to  read  on 
the  computer  (n=18,  10.29%). 

Each  of  these  reasons  for  printing  could  be  interpreted  as  a more  specific  indication  of  using  the  materials 
to  study.  They  each  support  the  literature  indicating  that  location  is  a barrier.  These  reported  reasons  for  printing 
also  support  the  literature  that  students  return  the  information  to  a format  in  which  they  are  familiar  (Pantel,  1997; 
Ruben,  1996)  that  is:  to  be  able  to  carry  the  information  with  them  as  they  do  text  books,  handouts  and  notes;  to  be 
able  to  access  the  information  in  class,  between  class,  and  in  multiple  locations;  and  to  be  able  to  write  on  the 
information. 

Learner  Investment  in  Time  on  Course  and  Printing 

The  average  time  students  invested  in  the  course  was  4.67  hours  (SD  = 3.96).  The  average  time  students 
invested  in  printing  was  20  minutes  (SD  = 25.94).  The  average  time  students  spent  printing  was  4%  of  the  average 
total  time  they  reported  spending  on  the  course.  In  addition,  only  three  participants  indicated  the  time  it  took  to  print 
as  a reason  not  to  print.  Therefore  the  time  it  takes  to  print  out  online  class  information  did  not  appear  to  be  a major 
factor  in  this  study. 
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Prior  Experience 

It  could  be  expected  that  the  more  prior  online  course  experience,  the  fewer  pages  printed.  It  does  not 
appear  that  prior  experience  in  learning  via  WBI  or  Web-enhanced  instruction  has  an  affect  on  the  printing  activities 
of  students  that  participated  in  the  study.  Ninety-nine  participants  (48%)  reported  that  this  was  the  first  course  they 
had  taken  that  had  a Web  component  (mean  = 1.96,  SD  = 1.09).  Thirty-nine  participants  (19.02%)  reported  they  had 
taken  one  prior  course  with  a Web  component  and  the  same  number  reported  two  prior  courses.  Twenty-five 
participants  (12.20%)  indicated  they  had  completed  three  or  more  courses  with  Web  components. 

Based  on  the  number  of  pages  printed  by  the  students,  further  analysis  was  needed  to  determine  if  this 
printing  activity  was  related  to  lack  of  experience  with  learning  online.  Using  Spearman’s  Rho  indicated  that  there  is 
a low,  positive  relationship  between  number  of  prior  courses  and  the  number  of  pages  printed  ( r = .130).  This  result 
indicated  students  with  more  prior  experience  in  courses  with  a Web  component  printed  as  much  as  or  slightly  more 
than  students  with  little  or  no  prior  experience.  Therefore  designers  of  WBI  can  not  assume  that  students  will  get 
use  to  using  course  materials  online  and  will  not  want  or  need  to  print  them. 

Influence  on  Satisfaction 

The  second  research  question  determined  if  students'  printing  activity  affected  their  satisfaction  with  the 

course. 

Difficulty  Using  the  Information  on  the  Course  Web  Site  and  in  Print 

The  majority  of  the  participants  (84.88%)  indicated  that  it  was  moderately  easy  or  very  easy  to  use  the 
information  on  the  Web  site  (mean  = 3.26,  SD  = .803).  Of  those  who  did  print  from  the  Web  site,  90.62%  reported 
it  was  moderately  or  very  easy  to  use  the  information  they  printed  (mean  = 3.20,  SD  = 1.01).  When  asked  what 
dificulties  student  has  with  printed  information,  fifty-one  of  the  participants  did  not  respond  to  this  question,  44 
participants  stated  that  they  did  not  have  difficulties,  and  18  reported  that  the  question  was  not  applicable.  The 
reported  difficulties  were  related  to  the  format  of  the  information  to  be  printed.  Examples  included  graphics  that 
would  not  print  or  were  not  legible  if  they  did  print  and  that  the  format  of  the  information  printed  was  ruined  making 
it  difficult  to  find  information. 

Satisfaction  with  the  Overall  Course 

Of  the  students  who  participated  in  this  study  82  or  40%  reported  they  were  mildly  satisfied  with  the  course 
and  41  or  20%  reported  they  were  very  satisfied  with  the  course  (mean  = 2.60,  SD  = 1.04).  Thirty-five  percent  of 
the  participants  indicated  that  their  printing  activity  mildly  increased  their  satisfaction  with  the  course  and  19% 
indicated  that  their  printing  activity  greatly  increased  their  satisfaction  with  the  course  (mean  3.63,  SD  = .996). 

Printing  and  Its  Affect  on  Satisfaction 

Spearman’s  Rho  was  used  to  determine  the  relationship  between  overall  satisfaction  with  the  course  and 
students'  perception  that  printing  affected  satisfaction  with  the  course  (r  = .334).  This  moderately  positive 
relationship  suggests  that  the  more  students  believed  that  printing  increased  their  satisfaction  with  the  course,  the 
more  they  perceived  that  they  were  satisfied  with  the  overall  course.  To  further  examine  how  students'  printing  was 
related  to  course  satisfaction,  Spearman’s  Rho  was  used  to  determine  the  relationship  between  the  number  of  pages 
printed  and  perception  that  printing  affected  satisfaction  for  the  course  (r  = .304).  The  results  indicated  that  the 
number  of  pages  printed  has  a low,  positive  relationship  to  the  affect  that  printing  had  on  student  satisfaction  for  the 
course.  Namely,  the  more  pages  participants  printed,  the  more  they  reported  an  increase  in  satisfaction  in  the 
course.  The  increase  in  satisfaction  based  on  printing  may  have  been  due  to  the  students’  ability  to  use  the 
information  they  printed  in  the  same  fashion  that  they  would  use  traditional  class  materials. 

Finally,  analysis  of  the  data  showed  that  satisfaction  for  the  course  and  students’  perception  that  printing 
activities  influence  satisfaction  were  not  related  to  where  students  reported  they  had  access  to  a computer  for  the 
course.  In  addition,  the  number  of  pages  printed  was  not  related  to  location  of  a computer.  In  other  words  whether  a 
student  used  a computer  at  home  or  in  a campus  lab  for  course  work  did  not  affect  their  perception  of  course 
satisfaction,  printing  activity,  or  the  number  of  pages  they  printed. 

Preference  for  Receiving  Learning  Materials 

This  finding  was  also  reflected  in  the  participants’  response  to  their  preference  for  receiving  the  materials. 
Fifty-nine  participants  (28.78%)  reported  that  they  would  prefer  the  option  to  purchase  a preprinted  packet.  Forty- 
nine  participants  (23.90%)  reported  that  they  would  prefer  to  have  a version  online  that  is  formatted  for  print.  Fifty- 
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two  participants  (25.27%)  reported  they  preferred  to  print  from  the  screen.  Thirty-three  of  the  participants  (16.10%) 
reported  that  they  preferred  to  use  the  information  online. 

Suggestions  for  Designers  and  Developers  of  WBI 

The  recommendations  reported  followed  three  themes.  The  first  theme  was  to  support  the  students  in  their 
printing  activities.  This  theme  included  recommendations  such  as  formatting  the  information  for  print,  breaking  the 
information  up  into  printable  chunks,  and  providing  printing  options  without  graphics  and  color.  The  second  theme 
was  related  to  the  actual  instruction.  This  theme  included  recommendations  such  as  instructors  still  needed  to  show 
enthusiasm  for  teaching  and  enhancing  communication.  The  third  theme  was  related  to  the  design  of  the  course 
Web  site.  This  theme  included  recommendations  such  as  providing  a study  guide  for  the  Web,  making  the  Web  site 
more  interactive,  and  ensuring  the  information  on  the  Web  was  accurate. 

Some  of  the  student  responses  did  not  apply  to  the  Web-based  aspects  of  courses  but  did  provide 
information  about  the  attitudes  and  beliefs  of  the  learners  who  participated  in  the  study.  Several  students 
commented  that  WBI  was  a positive  approach  to  learning.  “Online  courses  have  been  a blessing  and  are  absolutely 
a great  way  to  take  a class.”  “After  I got  used  to  using  the  web  site  I really  enjoyed  having  the  freedom  of  when  to 
get  my  work  done.”  Others  expressed  a very  different  attitude:  “Taking  quizzes  online  is  fine,  but  learning  online  is 
not.”  “Please  stop  making  Internet  classes,  they  are  just  another  fabrication  of  this  increasingly  impersonal  world 
and  is  seemingly  a waste  of  time.”  “I  think  that  having  the  entire  class  online  really  took  away  from  my  learning 
experience  and  I am  upset  for  having  to  pay  for  such  a format.”  “Personally,  I prefer  to  obtain  written  notes  in  a 
classroom  format  because  I think  that  the  instructor  and  students  interaction  is  very  important  to  really  learn  the 
material." 

The  Web  is  an  efficient  way  to  distribute  information  to  students  quickly  and  is  inexpensive  for  the 
faculty/institution.  Instructors  and  designers  need  to  be  aware  of  the  students’  perceptions  of  learning  online  and 
take  students'  concerns  into  consideration  when  developing  WBI. 

Implications  for  Design  and  Development  of  WBI 

The  information  obtained  in  the  study  presents  a number  of  implications  for  the  design  of  WBI.  These 
implications  support  the  instructional  design  principles  presented  in  the  literature  of  knowing  the  audience  and 
designing  effective  instructional  materials  to  meet  the  learners’  needs  and  preferences  (Dick  & Carey,  1990;  Gagne, 
Briggs  & Wager,  1992;  Reigeluth,  1999;  Seels  & Glasgow,  1997;  Smith  and  Ragan,  1993). 

First,  instructional  design  models  indicate  it  is  important  to  do  as  complete  an  analysis  as  possible  of  the 
audience  that  will  be  using  the  course  Web  site.  This  can  be  done  by  surveying  students  in  a traditional  offering  of 
the  class  or  in  a class  that  is  similar  to  or  a prerequisite  for  the  class  being  designed  for  the  Web.  If  the  students  are 
going  to  print  the  information  on  the  Web  site  to  facilitate  their  learning,  then  content  should  be  provided  in  a 
printable  format.  As  a number  of  students  indicated  in  their  recommendations,  they  want  to  be  able  to  print  the 
information  without  color  or  graphics. 

Secondly,  designers  should  look  more  closely  at  the  information  they  are  putting  online.  During  the  phase 
of  media  selection,  designers  need  to  determine  if  the  information  simply  replaces  another  medium  or  if  it 
supplements  information  presented  in  another  form.  Not  all  educational  experiences  are  appropriate  for  online 
technologies  (Druin  & Solomon,  1996;  Volker  & Simonson,  1995).  Questions  for  the  designer  to  investigate 
include:  Is  the  information  interactive,  providing  feedback  for  the  learner?  Does  the  information  present  some  form 
of  visual  communication  or  motion  that  can  not  be  presented  on  a black  and  white  sheet  of  paper?  Can  the  students 
easily  access  the  information  they  need  to  complete  assignments  and  study?  Answers  to  these  questions  influence 
decisions  to  place  course  information  online,  to  offer  it  in  a printable  version  or  to  offer  it  in  another  form. 

Thirdly,  the  designer  needs  to  consider  all  the  instructional  resources  available.  Students  did  not  express 
dissatisfaction  with  printing  information  from  the  course  Web  site.  If  the  textual  information  is  to  be  printed,  then 
both  course  designers  and  students  can  save  much  time  and  energy  if  resources  are  not  used  to  format  for  the  screen 
and  make  text  information  look  appealing  on  a computer.  Instead,  designers  can  spend  valuable  time  and  resources 
developing  instructional  activities  that  use  the  interactive  and  visual  features  of  the  Web.  Course  information  that  is 
typically  printed  can  easily  be  placed  on  the  course  Web  site  in  a printable  format.  This  activity  supports  Knox's 
(1980)  belief  that  alternative  materials  that  differ  in  emphasis  and  complexity  result  in  program  flexibility  and 
responsiveness  to  learners.  This  allows  participants  the  option  to  select  materials  that  are  relevant  and  challenging 
to  them,  set  their  own  pace,  and  maintain  interest  through  variety.  Students  can  use  this  information  to  enhance  their 
learning. 


185 


192 


Finally,  the  designer  of  WBI  needs  to  provide  information  on  how  the  Web  site  should  be  used.  Pantel 
(1997)  suggests  that  when  the  size  of  the  nonlinear  information  space  is  unknown  it  can  be  problematic.  He 
recommends  including  a site  map  that  provides  guidance  for  the  user.  Although  this  information  was  not  solicited, 
five  students  made  the  recommendation  to  provide  instructions  for  printing  and  three  students  recommended 
providing  a study  guide  for  the  course  Web  site. 

Recommendations  for  Future  Studies 

Many  institutions  of  higher  education  believe  that  delivering  course  content  on  the  Web  is  one  way  they 
will  remain  competitive  in  the  educational  market.  Much  pressure  is  being  placed  on  faculty  to  incorporate  the  Web 
into  their  teaching  and  learning.  More  research  is  needed  to  determine  optimum  design  standards  for  developing 
Web  courses  that  enhance  student  learning. 

This  study  provides  support  for  the  anecdotal  evidence  indicating  that  students  print  information  available  on  a 
course  Web  site.  It  also  suggests  reasons  why  the  students  are  printing.  Additional  research  is  needed  to  gain  more 
specific  information  regarding  how  students  use  the  information  they  print  from  the  Web  site.  The  primary  reason 
reported  for  printing  was  ‘to  study’.  This  raises  the  question  of  what  it  means  when  a student  says  'to  study'.  An 
investigation  of  the  literature  on  how  students  study  is  warranted. 

Additional  research  is  also  needed  to  determine  if  satisfaction  with  online  learning  is  affected  differently 
when  the  course  Web  site  is  used  in  combination  with  other  course  materials  as  in  a Web-enhanced  course  versus  a 
completely  Web  delivered  course. 

Finally,  additional  research  is  needed  to  determine  if  courses  with  Web  sites  which  are  designed  to  be 
highly  interactive  provide  different  results  than  those  courses  with  Web  sites  which  simply  provide  content  in  the 
form  of  text  and  graphics.  Specifically,  does  the  increased  online  interaction  affect  students'  satisfaction  with  the 
course  and  is  this  affect  based  on  the  interaction  or  on  the  fact  that  increased  online  interaction  would  require  the 
students  to  spend  more  time  at  a computer  to  participate  in  class. 

Summary 

Pantel  (1997)  states  it  simply  when  he  says  that  one  significant  advantage  of  the  Web  is  that  it  causes  us  to 
expand  the  notion  of  a document  by  enabling  multimedia  documents.  Whereas  print  is  better  at  presenting  text, 
photography  better  at  presenting  images,  radio  better  at  presenting  sound,  and  television  better  at  presenting  video, 
the  Web’s  advantage  is  that  it  combines  all  of  these  media  reasonably  well  into  a single  package. 

When  students  reach  post-secondary  education,  they  have  spent  most  of  their  lives  devising  and  hopefully 
perfecting  an  effective  way  to  learn  the  material  presented  in  their  classes.  When  faculty  place  their  course  material 
on  the  Web,  they  are  presenting  the  students  with  a new  challenge  in  learning.  Students  can  learn  the  mechanics  of 
accessing  a Web  site  and  navigating  through  it.  However,  it  takes  time  to  develop  new  study  habits. 

"Learning  as  a process  (rather  than  an  end  product)  focuses  on  what  happens  when  the  learning  takes  place” 
(Merriam  & Caffarella,  p.  124).  If  the  information  is  to  remain  online,  then  students  will  need  to  gain  new  tools  in 
how  to  use  the  information  effectively  while  it  remains  online.  This  requires  a change  in  students  study  habits. 
They  will  need  new  techniques  to  replace  taking  notes  on  the  printed  pages  and  highlighting  the  important 
information.  Until  new  study  habits  are  developed,  or  students  have  the  ability  to  take  notes  on  the  Web,  to 
highlight  online,  and  to  access  the  Web  remotely  with  pocket  size  computers  the  implication  for  the  design  and 
development  of  WBI  include  formatting  the  information  in  a way  that  facilitates  learning.  Students'  printing  activity 
is  an  important  consideration  for  the  design  and  development  of  Web-based  instruction. 
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The  questions. 


1.  What  is  the  name  of  your  course? 

2.  What  is  your  birth  date?  

3.  Please  indicate  your  gender. 

male 

female 

4.  Please  indicate  your  ethnic  origin. 

Asian 

Black,  non-Hispanic 

Hispanic 

American  Indian 

White,  non-Hispanic 

5.  Where  do  you  have  access  to  a computer  to  access  the  course  materials?  Select  all  that  apply. 

I use  a computer  with  Internet  access  at  home. 

I use  a computer  with  Internet  access  at  work. 

I use  a computer  with  Internet  access  in  a lab  on  campus. 

Other:  public  library,  friend,  etc. 

6.  Are  you  an  undergraduate  or  graduate  student? 

undergraduate 

graduate 

other 

7.  In  addition  to  participating  in  class  through  the  class  Web  site  do  you  meet  face-to-face  with  your  instructor  in  a 
classroom  setting. 

yes 

no 

8.  Do  you  print  out  pages  from  the  course  Web  site? 

yes 

no 

9.  If  you  do  not  print  out  pages  from  the  Web  site,  what  are  the  reasons  why? 

10.  How  many  pages  did/do  you  print  out  from  the  course  Web  site? 

0-5 

6-10 

11-20 

21-30 

31-40 

41-50 

more  than  50 

11.  What  type  of  information  was  available  on  the  course  Web  site?  Check  all  that  apply. 

Syllabus 

Course  schedule  or  calendar 

Online  discussion  content 

Assignments 

Content 
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12.  What  other  type  of  information  was  available  on  the  course  Web  site? 

13.  What  type  of  information  was  on  the  pages  you  printed?  Check  all  that  apply. 

Syllabus 

Course  schedule  or  calendar 

Online  discussion  content 

Assignments 

Content 

14.  What  other  information  available  on  the  course  Web  site  did  your  print? 

15.  Why  did  you  print  the  information  on  the  course  Web  site? 

16.  On  average,  how  much  time  do  you  spend  working  on  this  course  each  week? 

17.  On  average,  how  much  of  the  time  that  you  spend  working  on  this  course  is  spent  printing  from  the  Web  site? 

1 8.  Do  you  find  it  easy  or  difficult  to  use  the  course  information  on  the  course  Web  site? 

Very  difficult 

Moderately  difficult 

Moderately  easy 

Very  easy 

19.  Do  you  find  it  easy  or  difficult  to  use  the  course  information  you  print  from  the  Web  site? 

Very  difficult 

Moderately  difficult 

Moderately  easy 

Very  easy 

20.  If  you  do  have  difficulties  using  the  information  you  printed,  what  are  they? 

21.  Indicate  your  overall  satisfaction  with  this  course. 

very  unsatisfied 

mildly  unsatisfied 

mildly  satisfied 

very  satisfied 

22.  Does  printing  the  Web  pages  increase  or  decrease  your  satisfaction  for  the  course? 

greatly  decrease 

mildly  decrease 

no  affect 

mildly  increase 

greatly  increase 

23.  Indicate  the  number  of  courses,  that  require  you  to  access  a course  Web  site,  you  have  completed  prior  to  taking 
your  current  course. 

0 

1 

2 

3 or  more 

24.  Given  the  following  options,  which  would  your  prefer: 

To  purchase  a printed  packet  of  course  information  from  a bookstore 

or  similar  location? 

To  print  a version  of  the  information  formatted  for  the  printed  page. 

To  print  the  screens  as  they  appear  on  the  Web  site. 

To  use  the  information  online  and  not  print. 

25.  Do  you  have  recommendations  for  designers  and  instructors  of  online  course  information  regarding  your 
experiences 
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The  Relationship  Between  the  Types  of  Resources  Used  in  Science  Classrooms 
and  Middle  School  Students’  Interests  in  Science  Careers:  An  Exploratory 

Analysis 


The  U.S.  is  investing  millions  of  dollars  each  year  on  developing  new  resources  to  inspire  students  to 
pursue  science  careers,  yet  there  are  no  data  to  support  the  premise  that  specific  types  of  resources  are  associated 
with  student  interest  in  science  careers.  The  results  of  this  thesis  demonstrated  that  using  resources  high  in 
sociableness  and  high  in  webnicity  positively  and  significantly  related  to  higher  levels  of  interest  in  pursuing  science 
careers. 

Introduction 

Implications  for  future  applications  of  technology  are  many  ...  a mature  profession  recognizes  its  needs  and 
seeks  to  utilize  all  of  the  resources  - human  and  nonhuman  - that  it  can  muster  to  attain  its  goals  and  objectives  ... 
(Hayden  & Torkelson  1973,  p.  34). 

The  U.S.  invests  millions  of  dollars  each  year  on  securing  and  developing  new  resources  to  inspire  students 
to  pursue  science.  Since  middle  school  is  a time  when  students  begin  to  explore  careers  - and  select  and  de-select 
career  domains  to  pursue  (Gottfredson,  1981)  - it  is  important  to  understand  the  relationships  among  the  types  of 
resources  used  during  middle  school  science  and  the  level  of  student  interest  in  science  careers. 

There  are  indications  from  previous  research  that  different  sources  of  information  may  influence  a student’s 
choice  to  pursue  science  (Charron,  1991,  Jovanovic  & Steinbach  King,  1998;  Lederman  & Druger,  1985).  However, 
many  questions  remain  unanswered  in  regard  to  the  use  of  different  types  resources  in  the  classroom.  Is  using 
information  from  outside  the  classroom  - through  human  and  web  resources  - during  science  more  predictive  of 
students’  interest  in  science  careers  than  students  in  classrooms  that  do  not  regularly  employ  these  types  of 
resources?  Is  using  human  or  web  resources  during  science  class  predictive  of  boys’  science  career  interest  ...  of 
girls’?  To  find  answers  to  these  questions  data  was  collected  on  the  different  types  of  resources  used  in  the 
classroom  as  well  as  gender,  a factor  that  has  been  empirically  shown  to  be  related  to  the  development  of  career 
interests  (Hill  et  al.,  1990;  Hyde,  1993). 

Learning  and  Information  Resources 

Career  interests  are  learned  characteristics  (Super,  1984).  Instructional  designers  and  educators  have  an 
obligation  to  create  environments  that  facilitate  learning.  A key  step  in  the  instructional  design  process  is  developing 
or  integrating  existing  resources  that  will  provide  students  with  the  information  necessary  to  advance  their  learning 
process.  Not  much  is  really  known  about  the  relationship  between  learners  and  the  use  of  different  types  of 
information  resources  employed  in  the  classroom.  Decisions  about  resources  are  rarely  made  based  on 
understanding  the  relationships  between  learner-level  and  classroom-level  factors,  but  often  are  based  on  best 
guesses,  experience,  the  types  of  resources  that  are  available,  or  the  latest  and  greatest  resource. 

Dimensions  of  Informational  Resources 

In  this  study  the  types  of  supporting  classroom  resources  were  classified  along  two  dimensions  - 
“sociableness”  and  “webnicity”  of  information.  On  the  first  dimension,  “sociableness,”  a resource  is  defined  on  its 
measure  of  providing  information  through  interactions  with  people  (Sundar,  1994).  High  sociableness  occurs  when 
people,  human  resources,  are  actively  and  socially  involved  in  sharing  information  with  others  in  the  learning 
environment.  Books,  models,  pictures,  posters,  or  other  objects  are  considered  resources  low  in  sociableness.  Guests, 
experts,  and  others  who  participate  with  students  during  science  instruction  are  high  in  sociableness.  On  the  second 
dimension,  “webnicity”  (Grabowski,  personal  conversation  on  October  19,  1999)  is  a measure  of  a resource’s 
interconnectedness  of  information  to  other  supporting  information  on  a continuum  regardless  of  delivery  methods  or 
reference  to  the  form  of  information.  For  this  study,  on  one  extreme,  high  webnicity  refers  to  resources  providing 
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vast  and  easily  accessible  supportive  information  as  only  available  on  the  Internet.  On  the  other  extreme,  low 
webnicity  refers  to  resources  that  provide  limited  supporting  information  that  is  not  easily  accessed  in  time,  often 
classroom-bound,  i.e.,  requiring  students  to  leave  the  classroom  to  get  additional  information,  such  as  books, 
posters,  and  non-networked  computers.  Thus,  different  types  of  resources  provide  different  types  of  information  to 
students.  Much  of  the  literature  indicates  that  exposure  to  both  social  and  non-social  information  is  important  in 
developing  career  interests. 

Why  is  Studying  Student  Interest  in  Science  Careers  Important? 

Since  the  1950’s,  following  the  Russian  orbiting  of  Sputnik,  the  need  for  advances  in  science  education  has 
been  a concern  of  educators,  parents,  employers,  government  agencies,  and  the  American  society  at  large.  The 
government  undertook  actions  to  increase  the  scientific  literacy  and  interest  of  America’s  students  by  developing 
new  science  and  math  curricula,  educational  resources,  and  more  recently  expending  enormous  capital  to  supply  all 
classrooms  with  computer  and  Internet  technology.  This  movement  spawned  many  research  efforts  to  measure  the 
effects  of  these  actions  on  students’  pursuit  of  science  and  engineering  majors.  While  the  importance  of  well- 
prepared  curricula  to  help  educate  students  for  these  future  needs  is  recognized  by  many,  previous  studies  have  not 
investigate  whether  the  characteristics  of  the  classroom,  classroom  resources,  or  of  the  students  themselves  were 
related  more  strongly  to  the  development  of  interest  in  pursuing  science-related  careers. 

Discovering  the  relationships  among  classroom-level  and  student-level  characteristics  and  science  career 
interests  may  help  instructional  designers  and  educators  design  educational  environments  using  resources  that 
empirically  have  been  shown  to  be  predictive  of  the  development  of  student  interests  in  pursuing  science-related 
careers.  These  designs  would  then,  in  turn,  support  the  efforts  to  increase  student  interest  in  science  careers  in  the 
United  States. 

Development  of  Interest:  Interactions  and  Relationship  Building 

Interest,  an  acquired  attention  or  enthusiasm  for  a particular  field,  is  a learned  characteristic  and  has  been 
shown  to  be  the  key  factor  in  making  career  choices  (Super,  1984).  Developing  interest  in  a specific  career  domain, 
such  as  science,  is  a consequence  of  many  learning  interactions  with  the  people,  information,  and  objects  of  the 
practice.  Lave  & Wenger  (1991)  describe  learning,  such  as  developing  an  interest  in  pursuing  science,  not  as  a 
matter  of  a person's  internalizing  knowledge  but  as  a matter  of  a person's  transforming  his/her  participation  in  a 
social  community.  Learning,  therefore,  is  based  on  building  relationships  between  people  and  their  places  in  the 
community  and  associations  with  the  information  and  objects  of  the  practice.  Thus,  becoming  a member  of  a 
community  of  scientific  practice  requires  that  newcomers  interact  with  the  information,  artifacts,  and  people  that 
make  up  and  define  the  scientific  community. 

The  development  of  career  interests  is  a complex  phenomenon.  Much  of  the  research  provides  evidence 
that  there  are  specific  background  variables  a person’s  life  that  also  influence  the  development  of  specific  career 
interests.  When  children  enter  into  the  early  stages  of  adolescence  and  begin  to  explore  relationships  and  activities 
outside  the  family  they  begin  to  develop  independent  career  interests.  Interactions  with  reference  or  peers  groups, 
personality  characteristics,  societal  expectations,  perceived  abilities,  achievement  levels,  and  acquaintances  with  role 
models  outside  the  classroom  all  have  been  found  to  have  some  effect  on  the  career  aspirations  of  adolescents  (Hill, 
Pettus,  & Hedin,  1990). 

Development  of  career  interest  and  classroom  variables 

When  children  enter  into  adolescence  and  begin  to  develop  their  own  career  interests,  people  and  activities 
outside  the  home  and  specifically  in  the  classroom,  influence  them.  Over  the  last  two  decades,  much  research  has 
focused  on  investigating  the  relationship  between  rich  learning  environments  and  developing  interest  in  science 
careers,  especially  for  girls  and  minorities  (Hill,  Pettus,  & Hedin,  1990;  Kahle,  Matyas,  & Cho,  1985;  Mason  & 
Kahle,  1988).  However,  the  research  in  this  area  has  clearly  focused  on  the  results  of  sensitizing  teachers  to  the 
needs  of  minorities  and  training  teachers  in  the  use  of  sound  instructional  strategies.  No  published  literature  could  be 
found  that  reported  on  investigations  of  the  relationships  between  the  use  of  classroom  resources  and  student  science 
career  interests  in  typical  science  classrooms  as  opposed  to  classrooms  participating  in  science  intensive  programs. 

Most  theoretical  perspectives  also  acknowledged  the  interactive  effects  of  multiple  variables,  such  as 
resources,  instructional  strategies,  and  gender,  on  student  learning.  However,  a majority  of  the  educational  research 
neglects  to  take  into  account  the  influence  of  these  variables  on  developing  middle  school  students’  interest  in 
pursuing  science.  Nor  could  published  research  be  found  that  specifically  examined  the  predictive  relationship 
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between  the  use  of  different  types  of  informational  resources  in  science  classrooms  and  student  interest  in  pursuing 
science-related  careers. 

Social  Environment  of  the  Classroom 

A body  of  empirical  research  describing  the  relationship  between  psychological-social  aspects  of  the 
classroom  and  students’  interest  in  science  is  growing.  Evidence  exists  to  support  the  claim  that,  on  average, 
students  in  classrooms  with  strong  social  environmental  characteristics,  such  as  high  involvement,  peer  affiliation, 
and  competition,  do  manifest  more  positive  attitudes  toward  science  than  do  students  in  classroom  with  low 
measures  of  these  social  characteristics  (Fouts  & Myers,  1992;  Gallagher,  1994;  Kahle,  Matyas,  & Cho,  1985; 
Talton  & Simpson,  1987;  Yager  & Yager,  1985).  However,  Roth  (1996)  found  that  the  diffusion  of  the  various 
forms  of  knowledge  occurred  at  different  rates  and  in  different  forms  in  early  adolescent  student  groups.  Roth 
suggested  that  availability  of  specific  resources  and  social  networking  that  occurred  during  the  classroom  activities 
influenced  student  learning.  It  was  unclear  if  learning  was  related  more  strongly  to  the  webnicity  or  the  sociableness 
of  the  supporting  resources. 

Why  would  Sociableness  of  Resources  make  a Difference? 

Lent  et  al.  (1994)  argued  that  over  the  course  of  adolescence,  being  exposed  to  social  interactions 
differentially  reinforced  the  individual  for  pursuing  certain  activities  from  among  those  that  were  possible,  thus  gave 
rise  to  interests  in  specific  career  domains.  Short-term  social  interactions  and  relationships  with  resources  high  in 
sociableness  such  as  adults  outside  the  family  have  the  potential  to  cause  changes  in  an  adolescent’s  interests.  These 
types  of  relationships  do  not  generally  yield  emotionally  laden  feedback,  as  one  would  expect  from  family  or  close 
peers,  but  the  feedback  is  perceived  more  as  providing  a valid  evaluation  of  the  adolescent’s  behavior  (Darling, 
Hamilton,  & Niego,  1994).  Thus,  an  adolescent’s  self  image  can  be  influenced  by  resources  high  in  sociableness 
such  as  a significant  other  outside  the  family  because  the  adolescent  believes  that  the  other  is  a credible  source  of 
information  about  his/her  behaviors.  Exposing  early  adolescents  to  a variety  of  human  resources  during  science  class 
could  be  most  beneficial  in  fostering  curiosity  and  exploratory  behaviors  that  promote  career  interests  (Super,  1984). 

Why  would  Webnicity  of  Resources  make  a Difference? 

The  interconnected  nature  of  the  Internet  is  making  access  to  more  information  and  more  people  much 
easier  in  the  classroom.  The  types  and  numbers  of  resources  that  can  be  used  in  the  classroom  are  increasing  as 
schools  become  wired  and  access  to  the  Internet  expands.  Studies  have  found  that  when  web  resources  were 
introduced  into  the  classroom,  students  interacted  in  more  complex  tasks,  developed  greater  technical  skills,  and 
used  more  outside  information  (Hardin  & Ziebarth,  1995;  Owston,  1997;  Rice,  McBride,  & John,  1998)  than  before 
the  Internet  was  available.  Thus,  resources  high  in  webnicity  provided  access  to  additional  information  resources 
and  promoted  the  use  of  additional  resources. 

Web  resources  also  provide  connections  to  people  outside  the  classroom.  Research  in  this  area  reports  that 
collaborative  instructional  strategies  employing  web  resources  high  in  sociableness  and  encouraging  group  sharing 
and  knowledge  development  increased  student  achievement,  perceptions,  and  self-esteem  (Conlon,  1997;  Federman 
& Edwards,  1997;  Karayan  & Crowe,  1997;  Papert,  1997).  Central  to  those  studies  were  the  instructional  strategies 
used  to  encourage  interactions  among  students  and  others  outside  the  classroom.  It  is  not  known  whether  the  access 
to  resources  high  in  sociableness  or  the  instructional  strategy  itself  was  related  to  increases  in  the  learning  outcomes. 

What  is  not  currently  understood  is  whether  there  is  a significant  relationship  between  developing  career 
interests  in  science  and  the  use  of  resources  with  different  combinations  of  sociableness  and  webnicity.  Thus,  if 
students’  interests  in  science  are  promoted  when  they  are  actively  involved  with  information  and  people  (Super, 
1984)  and  they  access  more  supporting  information  when  using  web  technology  than  when  not,  then  unpacking  the 
relationships  between  the  types  of  supporting  information  resources  used  in  the  classroom  and  student  interests  may 
help  to  determine  which  types  of  resources  could  be  used  in  science  classrooms  to  inspire  students  to  pursue  of 
science  careers. 

Research  Methodology 

A one-time  cross-sectional  observational  method  was  used  to  collect  data  from  more  than  600  middle 
school  students  at  the  end  of  the  school  year  (Koszalka,  1999).  Student  data  were  collected  using  a modified  version 
of  the  Self-Directed  Search  Career  Explorer  for  Middle  School  students  (Holland  & Powell,  1994)  and  teacher 
questions  drawn  from  research  on  integrating  technology  resources  into  the  classroom  (Grabowski  & Koszalka, 
1998). 
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A snowball  sampling  procedure  was  used  to  identify  middle  school  science  teachers  who  would  be  willing 
to  participate  in  this  study.  Teachers  were  not,  however,  required  to  have  completed  any  special  teaching 
certification  programs  or  be  regularly  using  any  specified  procedures,  materials,  or  technologies.  All  classrooms 
were  required  to  have  school  access  to  Internet  technology  resources,  although  there  were  no  requirements  that 
teachers  had  to  regularly  use  these  resources.  The  students  who  participated  in  the  study  were  intact  groups  from  the 
science  classrooms  taught  by  the  participating  teachers.  •, 

The  dependent  variable  for  this  study  was  Science  Career  Interest . The  science  career  interest  measurement 
scale  was  continuous  (interval/ratio)  with  possible  scores  ranging  from  0 to  36.  The  higher  the  score,  the  more 
interest  the  student  had  in  pursuing  science-related  careers. 

The  student-level  independent  variable  for  this  study  was  gender  that  has  been  shown  to  be  related  to  an 
adolescent’s  interest  in  science  careers  (Boone  & Butler  Kahle,  1998;  Hill  et  al.,  1990). 

Teachers  were  asked  to  provide  information  on  the  types  of  resources  used  regularly  in  the  classroom. 
Classrooms  were  classified  into  resource  use  types  based  on  an  indication  of  whether  the  resources  had  been  used 
regularly  during  science  class,  i.e.,  5 or  more  times  during  the  school  year  (Becker,  1998).  Classrooms  were  coded 
into  only  one  of  four  resource  types: 

1 . Low  sociableness  and  low  webnicity-use  neither  human  or  web  resources  5 or  more  time  per  year, 

2.  High  sociableness  and  low  webnicity-use  human  resources  5 or  more  times  per  year  and  web  resources  less 
than  5 times  per  year, 

3.  Low  sociableness  and  high  webnicity-use  human  resources  less  than  5 times  per  year  and  web  resources  5 or 
more  times  per  year, 

4.  High  sociableness  and  high  webnicity-use  human  and  web  resources  5 or  more  times  per  year. 

At  the  beginning  of  a typical  science  class,  teachers  provided  students,  who  had  received  parental  consent, 
with  a paper  and  pencil  survey  of  their  career  interests.  The  teacher  collected  the  completed  surveys  and  signed 
consent  forms  and  returned  them  with  completed  teacher  surveys  in  a provided  mailer. 

All  data  used  for  analysis  were  first  summarized  to  provide  a description  of  the  participating  classrooms 
and  students.  All  descriptive  data  were  computed  using  SPSS  version  8.0.  The  complex  nature  of  the  multilevel 
environment  of  the  classroom,  where  students  are  nested  in  classrooms  and  classrooms  are  nested  in  schools,  can 
produce  inaccurate  measures  of  relationships  unless  the  characteristics  of  both  the  individual  student  and  the  groups 
of  students  are  taken  into  account  (Arnold,  1992).  Thus,  hierarchical  linear  models  (HLM)  were  used  to  examine  the 
associations  among  classroom-level  factors,  student-level  factors,  and  science  career  interest  using  two-level 
hierarchical  linear  models.  All  estimates  were  calculated  at  a .05  alpha-level.  Pair-wise  deletion  was  used  when 
missing  data  was  encountered. 

Results 

A total  of  658  surveys,  from  23  teachers  in  9 schools  were  administered  and  returned.  Fifty-eight  surveys 
were  either  returned  without  signed  parental  consent  forms  or  with  incomplete  data  and  were  removed  from  the 
sample.  The  remaining  619  surveys,  94%  of  the  returned  surveys,  were  included  in  the  data  analysis  that  included 
51%  girls  (n=304)  and  49%  boys  (n=297).  Eighteen  students  in  the  sample  did  not  identify  their  gender. 

This  sample  of  students  had  a fairly  high  level  of  interest  in  pursuing  science  careers  (M=23.47,  SD=5.37). 
The  scores  spanned  34  points  ranging  from  a minimum  score  of  2 to  maximum  score  of  36.  The  distribution  of 
scores  had  a slightly  negative  skew,  -.437  indicating  a slightly  higher  frequency  of  scores  above  the  mean  than 
below.  The  schools  represented  East  and  West  Coast  middle  class  suburban/rural  middle  schools. 

The  mean  science  career  interest  for  students  in  classrooms  that  used  resources  low  in  sociableness  and  low 
in  webnicity  was  17.46,  for  boys  it  was  18.10  and  for  girls  it  was  16.46.  The  mean  score  for  students  in  classrooms 
that  used  resources  low  is  sociableness  and  high  in  webnicity  was  21 .95,  for  boys  the  mean  score  was  20.12  and  for 
girls  it  was  24.19.  Students  in  classrooms  that  used  resources  high  in  sociableness  and  low  in  webnicity  had  a mean 
science  career  interest  score  of  20.68,  for  boys  the  mean  score  was  21.83  and  for  girls  it  was  21.52.  For  classrooms 
that  used  resources  both  high  in  sociableness  and  high  in  webnicity  the  mean  science  career  interest  score  was  28.33, 
for  boys  it  was  23.44  and  for  girls  it  was  29.3 1 . (See  Table  1 .) 
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Table  1.  Science  Career  Interest  and  Combinations  of  Sociableness  and  Webnicity 


Resource  Types 

All  Students 
M SD 

N 

Bovs 

M 

SD 

N 

Girls 

M 

SD 

n 

Low  Sociableness  and 
Low  Webnicity 

17.46 

6.05 

68 

18.10 

7.55 

30 

16.46 

4.44 

37 

Low  Sociableness  and 
High  Webnicity 

21.95 

4.67 

105 

20.12 

5.26 

53 

24.19 

3.69 

48 

High  Sociableness  and 
Low  Webnicity 

20.68 

5.74 

88 

21.83 

5.55 

46 

21.52 

6.01 

42 

High  Sociableness  and 
High  Webnicity 

28.33 

4.66 

358 

23.44 

4.88 

168 

29.31 

4.50 

177 

Notes:  Possible  range  for  science  career  interest  scores  is  0 — 36  points;  All  students:  n=619,  mean  score  23.47; 

Boys:  n = 297,  mean  score  23.7 1 ; Girls:  n = 304,  mean  score  23. 1 2 

An  HLM  baseline  analysis  of  the  data  revealed  that  the  average  mean  science  career  interest  score  across 
all  classrooms  was  23.63.  The  reliability  measure  for  this  data  was  .871,  indicating  that  the  sample  means  tend  to  be 
quite  reliable  as  indicators  of  the  true  classroom  means.  The  analysis  resulted  in  a chi  square  of  196.581  with  22 
degrees  of  freedom  and  a p < .000,  indicating  that  there  was  evidence  of  significant  difference  in  science  career 
interest  across  U.S.  middle  school  classrooms,  thus  a null  hypothesis  of  no  differences  in  science  career  interest  was 
rejected. 

When  classrooms  used  web  resources  during  science  lessons  students  had,  on  average,  a 3.86  point  higher 
score  in  science  career  interest  than  students  in  classrooms  that  did  not  employ  web  resources  (t=3.615,  p<.002). 
Similarly,  there  was  a significantly  higher  score,  on  average,  of  2.68  points  in  student  science  career  interest  in 
classrooms  that  used  human  resources  as  compared  to  science  classrooms  that  did  not  use  human  resources  (t=2.959, 
p<008).  (See  Table  2.) 

The  interaction  of  gender  and  human  resources  resulted  in  a positive  estimated  coefficient  (t  = 1.788,  p < 
.088)  indicating  that  human  resources  were  potentially  important  predictors  of  science  career  interest  for  boys.  The 
interaction  of  gender  and  web  resources  resulted  in  a negative  estimated  coefficient  (t  = -1.947,  p < .055).  Since 
boys  were  coded  as  “1”  and  girls  as  “0,”  the  negative  coefficient  indicated  that  the  use  of  web  resources  was 
potentially  a more  important  predictor  of  science  career  interest  for  girls.  Further  analysis  was  required  to  determine 
whether  the  use  of  human  and  web  resources  were  indeed  significant  predictors  of  science  career  interest  for  boys 
and/or  girls. 


Table  2.  Classroom-Level  Resources  and  Gender 


Fixed  Effects 

Estimated 

Coefficient 

Standard 

Error 

T-Ratio 

P-Value 

Science  Career  Interest  Mean 

23.53 

.43 

54.260 

0.000 

Use  of  Human  Resources 

2.68 

.91 

2.959 

0.008 

Use  of  Web  Resources 

3.86 

1.06 

3.615 

0.002 

Gender  slope 

0.21 

.42 

0.499 

0.623 

Gen  X Hum  Resources 

1.58 

.88 

1.788 

0.088 

Gen  X Web  Resources 

-2.87 

.91 

-1.947 

0.055 

Note:  N = 619,  Model  1 Intercept  Reliability  Estimate  = .732 


Data  were  split  by  gender  to  investigate  the  relationships  further.  The  mean  science  career  interest  score  for 
boys  was  23.69.  The  use  of  human  resources  was  a significant  predictor  of  science  career  interest  for  boys.  When 
human  resources  were  used  in  the  science  classroom,  boys,  on  average,  scored  2.68  points  higher  than  boys  in 
classrooms  that  did  not  use  human  resources  (t=2.187,  p<048).  Using  web  resources  (t=1.139,  p<  .269)  did  not 
significantly  predict  science  career  interest  for  boys.  The  use  of  both  human  and  web  resources  (t=.495,  p<626)  also 
did  not  significantly  predict  science  career  interest  for  boys.  (See  Table  3 and  Figure  1.) 
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Table  3.  Classroom-Level  Resource  Effects  for  Boys  and  Girls 


Fixed  Effects 

Estimated 

Coefficient 

Standard 

Error 

T-Ratio 

P-Value 

BOYS* 

Science  Career  Interest  Mean 

23.69 

.54 

44.017 

0.000 

Use  of  Human  Resources 

2.68 

2.26 

2.187 

0.048 

Use  of  Web  Resources 

2.14 

1.88 

1.139 

0.269 

Human  x Web  Resources 

1.29 

2.61 

0.495 

0.626 

GIRLS** 


Science  Career  Interest  Mean 

23.34 

.42 

55.207 

0.000 

Use  of  Human  Resources 

4.54 

1.78 

2.555 

0.020 

Use  of  Web  Resources 

7.06 

1.43 

4.941 

0.000 

Human  x Web  Resources 

4.63 

2.06 

2.252 

0.036 

Note:  * Boys:  N = 297,  Model  2 Intercept  Reliability  Estimate  = .597 

**  Girls:  N = 304,  Model  2 Intercept  Reliability  Estimate  = .5 14 


When  human  resources  were  used  in  science  class  girls,  on  average,  scored  4.5  points  higher  (t=2.555, 
p<.020)  in  science  career  interest  than  girls  in  classrooms  that  did  not  use  human  resources.  In  addition,  girls  scored, 
on  average,  7.1  points  higher  (t=4.941,  p<.000)  in  science  career  interest  when  web  resources  were  used  in  the’ 
classroom  than  girls  in  classrooms  that  did  not  use  web  resources.  When  both  web  and  human  resources  were  used 
in  science  class,  the  interaction  effect  of  both  produced,  on  average,  an  increase  of  16.2  points  (t=2.252,  p<.036)  in 
science  career  interest  for  girls  — interaction  effect:  4.5*1  +7.1*1  +4. 6*1*1  = 16.2. 

Boys’  Mean  score  by  Girls’  Mean  score  by 

Sociableness  Sociableness 


Sociableness  Sociableness 


Figure  1.  Science  Career  Interest  Scores  for  Boys  and  Girls  in  Science  Classrooms  High  and  Low  in  Webnicity,  by 
Sociableness 


Discussion,  Practical  Implications,  and  Future  Research 

Developing  interest  in  specific  career  domains  is  a consequence  of  many  learning  interactions  with  the 
people,  information,  and  objects  of  the  practice  (Lave  & Wenger  1991).  Conceptually,  previous  research  provided 
indications  that  working  with  science  practitioners  and  exploring  science  information  was  important  in  developing 
interests  in  science  careers  (Vondracek,  1993;  Helwig,  1998;  Hill  et  al.,  1990).  The  use  of  human  and  web  resources 
during  science  can  provide  adolescents  with  opportunities  for  exploring  science  by  providing  access  to  additional 
social  and  supportive  information.  Thus,  the  significant  relationships  found  support  a conceptual  hypothesis  that 
increasing  the  richness  of  information  through  human  and  web  resources  during  science  class  was  related  to  higher 
levels  of  science  career  interest.  (See  Figure  2) 
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Figure  2.  Level  of  Information  Richness 

Science  career  interest  was  predicted  differently  for  boys  and  girls  given  the  types  of  resources  used 
regularly  in  science  class.  It  was  hypothesized  that  using  web  resources  would  support  boys’  needs  to  manipulate 
and  explore  information  using  inanimate  resources,  yet  not  be  significantly  related  to  girls’  science  career  interest 
because  girls  were  thought  to  prefer  social  and  cooperative  learning  environments  (Schram,  1996).  Thus,  web 
resources  were  expected  to  be  predictive  of  boys’  and  human  resources  were  expected  to  be  predictive  of  girls’ 
science  career  interest.  Contrary  to  the  expectations  of  this  study  and  the  literature  on  gender  differences,  the  use  of 
resources  high  in  sociableness  was  predictive  of  higher  science  career  interest  for  both  girls  and  boys.  The  use  of 
resources  high  in  webnicity  had  a significant  predictive  relationship  for  girls’  interest  in  science  careers,  but  did  not 
predict  boys’  interests.  The  literature  on  gender  differences  in  educational  environments  theorizes  that  girls  learn 
better  in  socially  active  settings.  As  young  children,  girls  were  inducted  into  relational  modes  in  which  they  were 
encouraged  to  be  conscious  of  others’  needs  and  be  more  cooperative,  whereas  boys  were  encouraged  to  be 
autonomous  and  to  achieve  a sense  of  identify  on  their  own  (Boaler,  1997;  Head,  1996). 

Hyde  (1993)  cited  environmental  factors  such  as  cooperative  teaching  strategies  and  significant  people  in 
the  learning  locale  as  influencing  girls.  Classrooms  that  encouraged  independent,  hands-on  experimental  activities 
influenced  boys.  Girls  were  more  interested  in  animate  objects  high  in  sociableness,  especially  people,  than 
inanimate  objects  and  boys  were  found  to  be  more  interested  in  inanimate  objects,  low  in  sociableness,  more  than  in 
people.  Why  then,  did  boys  who  were  exposed  to  resources  high  in  sociableness  in  the  science  classroom  have 
higher  science  career  interests  than  boys  who  were  not?  And,  why  would  using  web  resources,  e.g.,  inanimate 
manipulatives,  not  be  predictive  of  boys’  interest  in  science  careers  and  be  a significant  predictor  for  girls’? 

Significant  others,  such  as  visitors  to  the  science  classroom,  influence  adolescents  both  through  the 
interactions  with  them  and  through  their  ability  to  serve  as  role  models  (Darling  et  al.,  1994).  A visitor  to  the  science 
classroom  may  provide  boys  with  a solid  understanding  of  a complex  scientific  concept  about  which  the  teacher 
lacks  expertise  or  engage  them  in  challenging  discussions  about  scientific  phenomena  to  increase  their  investigative 
skills.  Boys  may  identify  with  the  visitor,  especially  if  the  visitor  is  a male  and  fits  the  boys'  perceptions  of  being  a 
scientist  (Mason,  Kahle,  & Gardner,  1991),  thus  his  science  career  interest  could  be  significantly  influenced  by  the 
addition  of  the  human  resource  (Young,  Reynolds,  & Walberg,  1996).  Since  details  about  human  resources,  e.g. 
gender,  strategies  used  with  students,  etc.,  were  not  gathered  research  is  needed  to  confirm  these  explanations. 

It  was  also  surprising  to  find  that  using  web  resources  in  science  class  was  highly  predictive  of  girls’ 
science  career  interests  and  not  predictive  of  boys’.  The  use  of  human  resources  in  the  science  classroom  predicted 
higher  science  career  interest  in  girls;  however,  the  use  web  resources  predicted  girls’  science  career  interest  scores 
seven  points  higher,  on  average,  as  compared  to  girls  who  did  not  have  access  to  web  resources.  The  use  of  both 
web  and  human  resources  was  related  to  dramatically  higher  scores  in  the  girls’  science  career  interest.  Why  were 
girls’  interests  in  science  career  predicted  by  the  use  of  web  resources  and  boys’  not? 

Girls  are  generally  thought  to  learn  better  in  social  environments  and  are  expected  to  have  strong 
associations  with  resources  high  in  sociableness.  Girls  are  theorized  to  benefit  more  from  social  learning  activities. 
Much  of  the  research  illustrated  that  science  programs  focusing  on  girls  and  incorporating  cooperative  instructional 
strategies,  science  equipment,  and  other  science  materials  influenced  girls’  interest  in  science  (Mason  & Kahle, 
1988;  Schram,  1996).  However,  it  was  not  clear  whether  the  influence  was  from  the  social  nature  of  the  activities, 
use  of  equipment,  or  exposure  to  new  information.  Girls  could  potentially  be  intrigued  by  the  use  of  web  resources 
to  investigate  areas  they  have  not  been  encouraged  to  explore  previously. 

The  use  of  the  web  may  also  provide  another  mechanism  for  encouraging  social  interaction  in  science  class 
on  several  different  levels.  First,  the  low  number  of  computers  generally  found  in  classrooms  may  make  it  more 
effective  to  have  students  work  in  cooperative  groups  while  completing  science  projects  using  web  resources.  This 
type  of  strategy  enhances  the  social  nature  of  using  web  resources,  thus  could  explain  the  relationship  to  girls’ 
science  career  interests.  Secondly,  the  use  of  web  resources  in  the  classroom  may  predict  girls’  science  career 
interest  because  girls  reportedly  do  not  interact  with  computer  technology  outside  the  classroom  as  much  as  boys 
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(Rocheleau,  1995).  The  use  of  web  resources  may  have  provided  girls  in  this  study  with  a novel  experience  thus 
increasing  their  opportunities  and  interest  in  exploring  science  content  and  conversing  with  science  experts  virtually. 

Boys,  on  the  other  hand,  have  generally  been  shown  to  be  the  heavier  users  of  computer  technology  outside 
of  the  science  classroom  (Rocheleau,  1995).  Because  boys  are  thought  to  be  the  heavier  users  of  computer 
technology,  they  are  most  likely  to  be  more  knowledgeable  of  the  web  and  the  resources  it  has  to  offer.  Boys  with 
interests  in  science  are  therefore  more  likely  to  use  the  web  to  explore  science-related  sites  outside  of  class.  Using 
web-based  science  resources  inside  the  classroom  thus  may  not  be  unique  to  boys  thereby,  reducing  the  potential 
relationship  between  the  use  of  web  resources  in  the  science  classroom  and  their  interest  in  science  careers. 
Additionally,  if  students  are  mainly  assigned  to  cooperative  activities  when  using  web  resources,  boys,  who  prefer  to 
learn  autonomously  (Boaler,  1997;  Head,  1996),  may  not  be  benefiting  from  the  use  of  web  resources  in  the  science 
classroom  or  be  more  apt  to  explore  sites  not  related  to  science  assignments. 

Further  research  is  required  to  understand  the  circumstances  of  use  that  demonstrated  significant 
relationships  between  different  types  of  web  resources  and  science  career  interest  among  girls  and  boys.  Does 
increasing  the  amount  of  time  on-line  relate  to  higher  levels  of  science  career  interest?  What  types  of  web  resources 
promote  science  career  interest?  What  types  of  interactions  with  web  resources  promote  science  career  interest?  Is  it 
more  beneficial  to  work  in  a group  or  individually  during  activities  that  employ  web  resources? 

Understanding  the  relationships  between  the  use  of  different  types  of  resources  during  class  and  students’ 
science  career  interests  can  provide  a basis  for  developing  and  securing  resources  that  have  been  empirically  shown 
to  be  related  to  higher  interest  in  science  careers.  If  these  results  hold  up  in  replication  studies,  there  are  significant 
practical  implications  that  may  drive  educational  policy  decisions.  Given  the  goal  of  inspiring  students  to  pursue 
science  careers  and  the  relationship  between  interest  in  science  careers  and  use  of  human  and  web  resources,  policy 
makers  should  provide  financial  commitments  to  enhance  computer  equipment,  facilitate  access  to  science 
information,  and  develop  communication  and  working  opportunities  with  scientists  and  experts  in  science 
communities.  In  addition,  commitments  should  be  made  to  adequately  support  teachers  in  their  efforts  to  integrate 
these  resources  into  their  teaching  and  learning  environments.  Teachers  should  be  given  release  time  to  prepare 
lessons  that  incorporate  human  and  web  resources  and  in-service  training  that  focuses  on  developing  strategies  and 
skills  to  enhance  existing  lessons  with  web  resources.  These  efforts  would  help  to  increase  student  access  to  human 
and  information  resources  conducive  to  exploring  science. 

These  findings  shed  new  light  on  understanding  the  complex  relationships  between  the  use  of  resources  in 
the  classroom  and  multiple  factors  that  affect  the  development  of  science  career  interest.  Without  an  understanding 
of  the  relationships  between  the  types  of  resources  used  in  the  educational  environment  and  student  science  career 
interest  there  is  a risk  that  large  investments  in  educational  resources  will  go  unmatched  in  student  outcomes.  The 
results  of  this  study  demonstrated  that  using  resources  high  in  sociableness  and  high  in  webnicity  during  science  was 
positively  and  significantly  related  to  higher  levels  of  student  interest  in  pursuing  science  careers. 
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Action  Research  on  Training  Students  to  Be  Independent  Learners 

Amy  S.  C.  Leh 

California  State  University  San  Bernardino 


Abstract 

Technology  advancement  is  shifting  our  education  paradigm . The  role  of  the  instructor  is  changing  from 
an  information-giver  to  a facilitator.  Students  no  longer  passively  receive  information.  Given  opportunities, 
students  may  become  instructional  resource  providers  and  self-learners . This  article  reports  how  the  author 
encouraged  students  to  be  resource  providers  and  trained  them  to  be  self-learners  and  self-trainers  in  an 
educational  multimedia  course.  Class  observations  and  students’  feedback  revealed  that  the  teaching  methods  and 
new  role  of  the  instructor  had  positive  impact  on  students  ’ learning. 

What  is  a teacher?  Who  is  the  teacher?  Answers  to  these  questions  have  been  changing  in  the  past  few 
years  due  to  technology  advancement.  Years  ago,  teacher  was  the  main  information  giver  and  center  of  a classroom. 
Lately  modem  technology  has  been  shifting  the  education  paradigm  and  providing  students  with  information  from  a 
variety  of  resources  via  various  channels.  Consequently  the  teacher  is  no  longer  the  center  of  the  classroom. 
According  to  the  report  released  by  the  National  Council  of  Accreditation  of  Teacher  Education  (NCATE)  in 
September  of  1997,  teachers  should  develop  a new  understanding,  new  attitude,  new  approach,  and  new  role 
(NCATE,  1997).  Tom  Carroll,  director  of  Preparing  Tomorrow’s  Teachers  to  Use  Technology  (PT3)  grants  of  the 
US  Department  of  Education,  vividly  described  the  changing  role  of  teachers  at  the  annual  convention  of  Society  for 
Information  Technology  and  Teacher  Education  (SITE)  2000. 

Carroll  (2000)  advocated  that  a teacher  should  take  the  role  of  a learner.  A teacher  is  an  expert  learner 
while  a student  is  a novice  learner.  Novice  learners  may  become  expert  learners  as  time  progresses.  As  an  expert 
learner,  the  teacher  should  facilitate  learning  for  novice  learners  and  let  them  become  instructional  resource 
providers.  Creating  a learning  community  that  consists  of  expert  learners,  novice  learners,  parents,  and  members 
who  may  foster  the  learning  process,  a teacher’s  responsibility  is  to  facilitate  interaction  and  learning  within  the 
learning  community  and  to  further  expand  the  community  by  involving  members  of  other  communities. 

Hence  learning  no  longer  refers  to  learning  from  a teacher.  A student  may  learn  from  other  students  or  any  member 
of  the  community.  A student  who  does  not  know  one  area  may  be  expert  in  another  area.  Thus  a student  may  be  a 
receiver  at  one  time  while  he  (she)  may  be  a provider  another  time.  A student  may  also  self-learn  from  available 
resources,  for  example,  the  Internet.  Learning  becomes  multifaceted  and  dynamic,  and  resources  become  essential 
in  the  learning  process. 

In  addition  to  Carroll,  Oliver  (2000),  an  invited  speaker  at  World  Conference  of  Educational  Multimedia, 
Hypermedia,  and  Telecommunications  (EDMEDIA),  also  supported  the  idea  of  using  students  as  instructional 
resource  providers.  In  his  speech,  he  further  explained  how  he  treated  his  students  as  resource  providers  and  how 
his  students  contributed  to  the  teaching  materials.  Similar  ideas  were  echoed  at  two  presentations  at  the  convention, 
Leh  (2000)  and  Santema  & Genang  (2000).  The  author  of  the  paper  was  one  of  the  presenters  and  was  pleasantly 
surprised  to  see  the  idea  simultaneously  spring  up  on  three  different  continents,  North  America  (USA,  Leh),  Europe 
(Netherlands,  Santema  & Genang),  and  Australia  (Oliver).  This  notion  might  be  an  indicator  of  a current  global 
educational  trend. 

In  this  paper,  the  author  will  share  her  experience  why  the  role  of  instructors  in  Instructional  Technology 
has  to  change.  She  then  will  describe  how  she  played  the  role  of  a facilitator  and  encouraged  her  students  as 
educational  resource  providers  in  an  Instructional  Technology  course.  She  will  also  report  her  students’  opinions 
towards  their  learning  in  the  course. 

The  Need  for  the  Changing  Role 

The  author  is  a university  professor  who  teaches  technology  credential  and  graduate  courses  in 
Instructional  Technology  at  the  College  of  Education  of  a public  university  in  the  USA.  The  students  of  the  course 
were  K12  schoolteachers  working  on  their  master’s  degree  at  the  university.  Due  to  state  mandate,  her  students  had 
to  integrate  technology  into  their  classroom.  In  addition,  because  they  were  master’s  students  in  Instructional 
Technology,  they  were  also  expected  to  help  their  colleagues  in  their  schools  to  employ  technology.  These  two 
forces  generated  their  need  to  study  the  use  of  technology,  especially  a variety  of  computer-based  software 
applications  and  existing  technology  resources. 
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How  could  the  author  successfully  and  effectively  help  her  students?  She  realized  that  technology 
advancement  had  changed  how  people  learned  and  altered  the  role  of  the  instructor.  She  could  no  longer  be  an 
information  giver  but  had  to  become  a learner  learning  with  her  students.  For  example,  while  the  author  was  a 
student,  she  learned  HyperCard,  a commonly  used  multimedia  authoring  program  during  that  time.  When  she 
became  an  instructor,  HyperCard  was  considered  to  be  an  out-of-date  program  and  most  people  no  longer  used  the 
program.  Similarly,  while  she  was  a student,  only  professionals  developed  webpages.  Now  webpage  development 
has  become  common  knowledge  and  a necessary  skill  for  many  people.  She  could  not  depend  on  what  she  learned 
in  school  but  had  to  keep  learning  new  technology  by  herself.  Likewise,  her  students  would  face  the  same 
challenges.  They  had  to  become  self-learners  of  technology. 

Technology  integration  requires  people's  knowledge  and  expertise  of  subject  areas.  The  author  might  be 
an  expert  of  integrating  technology  into  one  subject  area,  like  second  language  learning,  but  might  not  know  how  to 
effectively  integrate  technology  into  another  subject  area,  for  example,  science.  As  a result,  the  author  had  to  learn 
with  her  students  who  were  experts  of  their  subject  areas. 

Due  to  the  two  facts  mentioned  above,  the  author  changed  her  role  to  be  a facilitator  and  hoped  to  set  a 
good  teaching  example  for  her  students.  She  encouraged  her  students  to  be  instructional  resource  providers  and 
trained  them  to  be  self-learners.  She  prepared  a learning  environment  in  which  students  could  learn  by  themselves 
and  from  each  other.  The  computer  applications  they  learned  in  her  classes  might  be  obsolete  one  day;  however,  the 
learning  skills,  she  hoped,  might  be  transferred  and  applied  to  new  learning  experiences.  Illustrated  below  are  her 
experiences  with  teaching  one  of  the  graduate  courses,  “Advanced  Computer  Applications  in  Education”. 

Students:  Resource  Providers  and  Self-Learners 

The  goal  of  the  course  “Advanced  Computer  Applications  in  Education”  was  to  familiarize  students  with  a 
variety  of  authoring  multimedia  software  programs.  Since  this  was  the  only  course  directly  dealing  with  such 
applications  in  the  academic  program,  the  author  structured  the  course  as  a multimedia  survey  course  in  which 
students  studied  a variety  of  applications  rather  than  focused  on  a specific  computer  program. 

She  assessed  students’  skills  at  the  beginning  of  the  course.  On  a survey,  students  identified  their  skills  of 
using  the  following  seven  computer-based  application  software:  Webpage  development  tool,  HyperStudio, 
PowerPoint,  PhotoShop,  Premier,  Authorware,  and  Director.  The  students  circled  one  of  the  following — “don’t 
know”,  “good”,  “very  good”,  and  “excellent” — which  best  described  their  skill  level. 

The  author  reported  the  survey  results  in  class.  She  intended  to  help  the  students  use  the  information  to 
complete  their  course  assignments:  developing  a HyperStudio  project,  creating  a webpage,  integrating  PowerPoint  in 
instruction,  and  offering  a technology  training  session.  Course  assignments  emphasized  integrating  technology  into 
content  areas,  independent  learning,  and  learning  from  each  other.  Described  below  is  how  the  students  learned 
from  each  other  in  the  “Technology  Training”  assignment. 

For  the  assignment,  the  students  selected  their  team  members  (no  more  than  three  members  in  a team). 
Each  team  chose  and  learned  one  of  the  software  applications  mentioned  above.  Overview  of  software  and  step-by- 
step  instruction  of  using  some  of  the  software  were  illustrated  in  a course  textbook. 

After  selecting  the  software  program,  the  students  constructed  a training  plan  for  a 60  to  90  minute  training 
session.  In  the  training  session,  they  were  supposed  to  teach  their  classmates  the  use  of  the  application  software  they 
selected.  The  training  plan  had  to  contain  (1)  assessment,  (2)  time  length,  (3)  content  outline,  (4)  evaluation,  and  (5) 
training  materials  like  handouts  or  evaluation  sheet  if  applicable.  They  met  with  the  author  to  discuss  their  training. 
They  had  to  be  prepared  to  answer  questions  like  what  prerequisite  skills  their  trainees  had,  what  they  would  cover 
in  the  training,  which  criteria  they  used  to  decide  on  the  content  in  the  training,  and  how  they  would  evaluate  the 
success  of  their  training  session.  The  students  were  aware  that  they  could  ask  for  help  from  the  instructor  (the 
author)  at  any  point  of  time. 

Finally  they  delivered  the  training  to  the  entire  class.  The  author  observed  the  class  and  recorded 
classroom  interaction.  She  also  took  notes  how  the  trainers  could  make  the  training  better.  After  the  training,  the 
author  first  asked  the  trainers  to  self-evaluate  their  training.  She  then  asked  the  trainees  to  critique  the  training 
session  by  providing  good  points  and  suggestions  for  improvement.  She  asked  the  trainees  to  provide  oral  input  in 
class  so  that  the  trainers  could  receive  instant  feedback  and  response.  She  conducted  the  discussion  and  concluded 
with  her  own  critiques.  The  trainers  also  received  written  feedback  afterwards  from  the  trainees  and  the  instructor. 
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Evaluation  and  Opinions  of  Students 

During  the  quarter,  the  author  observed  the  class  and  interviewed  students  about  their  learning  experiences 
in  the  course.  At  the  end  of  the  quarter,  the  author  interviewed  the  students  and  also  had  them  fill  in  a survey 
expressing  their  opinions  about  the  course.  The  responses  were  analyzed  and  categorized.  The  results  revealed  that 
the  teaching  methods  used  in  the  course  were  deemed  to  be  successful. 

The  teaching  methods  highly  motivated  the  students.  The  students  were  enthusiastic  about  choosing  a 
training  topic  and  being  actively  involved  in  the  learning  process.  Some  of  them  chose  software  they  were  familiar 
with  but  more  than  half  of  them  selected  software  that  they  knew  little  about.  When  asked  why  they  selected  the 
software  foreign  to  them,  the  students  answered  that  this  would  be  a good  opportunity  for  them  to  learn  new  skills. 
One  said,  “I  wanted  to  learn  this  software  for  a while.  Selecting  this  topic  may  push  me  to  learn  it.”  They  learned 
beyond  the  textbook.  Two  of  them  rented  a videotape  on  Macromedia  Director.  Three  of  them  bought  a book  and 
together  studied  Adobe  PhotoShop.  They  mentioned  that  they  enjoyed  learning  with  their  partners.  They  also 
expressed  their  frustration  when  they  explored  software,  for  example,  the  learning  curve  of  studying  a sophisticated 
software  application.  Despite  the  frustration,  they  cherished  the  learning  experience  and  thought  that  they  learned  a 
lot  from  their  peers  and  other  resources,  not  only  from  the  instructor. 

The  teaching  methods  generated  students’  meaningful,  active,  and  constructive  learning.  For  the  training 
plan,  the  students  constructed  their  own  instrument  to  assess  the  trainee’s  skills,  decided  on  training  content,  and 
determined  how  to  evaluate  their  training.  They  mentioned  that  instructional  design  models,  for  example  Dick  and 
Carey’s  model,  made  much  more  sense  when  they  went  through  the  process.  They  expressed  that  they  spent  much 
time  on  the  course,  much  more  than  what  they  spent  on  a regular  course.  However,  they  liked  the  experience.  They 
felt  that  they  were  the  masters  of  their  learning  and  felt  sense  of  ownership.  They  mentioned  that  this  learning 
experience  would  influence  how  they  learn  and  how  they  teach  in  the  future.  They  would  search  for  resources  that 
could  foster  their  learning,  and  the  instructor  would  only  be  one  of  the  resources.  They  would  also  try  to  play  the 
role  of  a facilitator  rather  than  an  information  giver  in  their  classrooms  when  appropriate. 

The  author  required  the  students  to  critique  their  classmates’  training  sessions.  She  noticed  that  the 
students  could  easily  say,  “You  did  a great  job!”  Nevertheless,  they  had  difficulty  in  addressing  points  of  improving 
a training  session.  By  requiring  the  students  to  specify  good  points  and  provide  recommendations  for  improvement, 
they  practiced  to  think  critically.  At  the  end  of  the  quarter,  the  author  noticed  the  improvement  that  the  students 
made  on  critique. 

The  teaching  methods  increased  the  students’  self-confidence.  Before  taking  the  course,  the  students  often 
complained  that  they  did  not  know  how  to  use  certain  software,  for  example,  webpage  development  tool,  because 
the  instructors  did  not  teach  them.  The  students  seemed  to  count  on  the  instructors  and  did  not  feel  comfortable  of 
learning  by  themselves.  At  the  end  of  the  course,  the  students  expressed  that  the  course  increased  their  confidence 
in  learning  technology  on  their  own.  They  mentioned  that,  if  they  could  learn  software  with  their  partners  and 
successfully  provide  training  to  their  classmates  in  this  course,  they  should  be  able  to  do  the  same  elsewhere.  They 
became  comfortable  of  being  self-learners. 

The  students  also  benefited  from  one  of  the  textbooks  used  in  the  multimedia  course.  Many  instructors 
could  not  include  advanced  multimedia  software  in  their  courses  because  institutions  could  not  afford  the  expensive 
software.  The  textbook  offered  a possible  solution  to  the  problem  that  educators  and  students  often  encountered. 
The  book  included  a CD  of  recent  multimedia  software — MacroMedia  Authorware,  MacroMedia  Director, 
MacroMedia  SoundEdit  16,  Adobe  PhotoShop,  and  Adobe  Premiere — and  step-by-step  instruction  on  the  use  of  the 
software.  The  advantage  of  such  a book  was  that  users  could  explore  and  learn  the  software  at  low  cost, 
approximately  $40  US  dollars.  They  could  also  independently  practice  the  software.  The  disadvantage  was  that 
saving  of  files  was  restricted.  Several  companies  published  books  similar  to  the  book  mentioned  above.  Because  of 
the  opportunity  of  exploring  a variety  of  advanced  multimedia  software  at  low  cost,  the  author  highly  recommends 
using  such  a book  to  make  students  learning  possible. 

Conclusion 

Technology  advancement  is  shifting  our  education  paradigm.  The  role  of  the  instructor  is  changing  from 
an  information-giver  to  a facilitator.  Students  no  longer  passively  receive  information  but  may  be  instructional 
resources  in  class.  Given  opportunities,  they  may  be  self-learners  and  self-trainers. 

In  a multimedia  course,  the  instructor  employed  teaching  methods  allowing  her  to  be  a facilitator  and  her 
students  to  be  self-learners.  It  was  found  that  the  course  motivated  students,  fostered  students’  active,  meaningful, 
and  constructive  learning,  encouraged  students’  critical  thinking  skills,  and  increased  students’  confidence.  Class 
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observations  and  students’  feedback  revealed  that  the  new  teaching  methods  and  role  of  an  instructor  had  positive 
impact  on  students’  learning. 

As  a university  professor  in  Instructional  Technology,  the  author  might  have  experienced  the  education 
paradigm  shift  and  its  impact  on  the  role  of  an  instructor  earlier  or  faster  than  instructors  of  other  subject  areas 
might.  However,  the  changing  role  is  a strong,  perhaps  inevitable,  trend.  Every  instructor  should  be  open  to  the 
idea  and  explore  the  possibility  and  experiment  with  the  opportunity. 

As  NCATE  stated  in  1997,  teachers  need  to  develop  a new  understanding,  new  attitude,  new  approach,  and 
new  role.  Every  instructor  should  be  open  to  the  changes  and  further  create  a learning  community  in  which 
instructors,  students,  and  community  members  may  contribute,  benefit,  and  generate  meaningful  learning 
experiences.  One  can  only  look  forward  to  participating  in  the  dynamic  learning  and  expect  its  positive  impact  on 
our  society. 
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Abstract 

It  is  a very  recent  phenomenon  for  gender  differences  to  receive  attention,  even  though  research  on  CMC 
goes  back  to  the  1970s,  to  an  earlier  stage  in  computer  technology.  The  current  research  reports  survey  results  on 
gender  differences  in  cyber  learning  strategies  with  respect  to  expression,  information  processing,  self  control,  and 
use  of  human  resources;  and  suggests  instructional  interventions  to  guarantee  a gender-free  cyber  learning 
environment. 

Introduction 

Distance  education  is  moving  toward  a new  generation,  which  is  the  Internet-based  cyber  education. 
Today,  functions  and  advantages  of  the  Internet  will  be  further  capitalized  for  education,  especially  learner-centered 
learning.  The  Gartner  Group  (1998)  estimates  that  60%  of  higher  education  students  will  access  content 
electronically  by  2003. 

Computer  mediated  communication  in  which  historically  males  continued  to  dominate  (Herring,  1994)  but 
increasing  numbers  of  female  populations  are  getting  benefits  from  this  new  technology.  Recent  reports  and  studies 
indicate  that  females’  access  to  the  Internet  is  substantially  improving  (Refer  to  ALMANAC,  1990;  Media  Metrix, 
1999).  However,  it  is  still  difficult  to  conclude  that  females  are  participating  equally  with  males  in  the  cyberspace. 
Although  especially  the  Internet  among  various  technologies  has  been  expanding  in  the  educational  community 
worldwide,  it  is  not  clear  whether  all  and  every  learner  will  benefit  equivalently  from  this  new  learning  technology. 
It  is  a very  recent  phenomenon  for  gender  differences  to  receive  attention,  even  though  research  on  CMC  has  been 
back  to  the  1970s,  to  an  earlier  stage  in  computer  technology  (Herring,  1994). 

Learner  issues  are  the  core  elements  for  success  in  cyber  education  (Bonk  & Dennen,  1999)  and  learning 
strategies  among  those  issues  are  especially  significant  factors.  Cyberspace  is  a highly  learner-centered  place,  where 
learners  should  take  more  responsibility  for  their  own  learning.  This  is  primarily  due  to  the  nature  of  cyberspace, 
which  is  represented  by  time  and  space  independence,  openness,  constructiveness,  and  webbness.  Therefore,  learner 
issues  are  the  core  element  for  success  in  cyber  education;  and  learning  strategies,  through  which  learners 
themselves  regulate  their  own  learning  process,  are  taking  a more  critical  part  than  instructional  strategies. 
Nonetheless,  the  existing  literature  on  learning  strategies  in  cyber  space  is  limited  and  none  of  it  focuses  on  gender 
differences.  The  current  research  reports  survey  results  on  gender  differences  in  cyber  learning  strategies  as  follows: 

(1)  How  do  genders  differ  in  ‘expression’  in  cyberspace? 

(2)  How  do  genders  differ  in  ‘information  processing’  in  cyberspace? 

(3)  How  do  genders  differ  in  ‘self-control’  in  cyberspace? 

(4)  How  do  genders  differ  in  ‘the  use  of  human  resources’  in  cyberspace? 

Literature  review 

Learner  characteristics 

A great  deal  of  distance  education  literature  supports  the  claim  that  learner  characteristics  are  very  critical 
factors  for  achievement  and  satisfaction  levels  in  distance  education  (e.g.  Willis,  1994;  Smaldino,  1999;  Simonson, 
1999).  Willis  (1994)  suggests  adult  distance  learners’  success  factors  including  tolerance  for  ambiguity,  a need  for 
autonomy,  and  an  ability  to  be  flexible;  and  failure  factors  including  preference  of  a great  deal  of  structure,  face-to- 
face  lectures  and  the  opportunity  to  interact  with  instructors. 

The  goal  of  planning  cyber  education  programs  is  to  provide  learning  experiences  and  environments  based 
on  the  each  learner’s  needs  so  to  ‘makes  the  sum  of  each  learner  equivalent’  (Simonson,  1999)  by  incorporating  the 
proper  design  elements  and  instructional  strategies. 

Cyberspace  and  learning  strategies 

Substantial  numbers  of  literature  lead  us  to  the  conclusion  that  learning  strategies  take  the  important  roles 
for  learning  performance  (Dansereau,  1978;  Weinstein,  1978).  Ignoring  learning  strategies  is  basically  discouraging 
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learners  from  developing  and  exploring  their  own  new  learning  strategies.  Nonetheless,  relatively  insufficient 
attention  to  learning  strategies  is  found  in  the  current  cyber  education  literature.  Burge  and  Eastmond  found  that 
learners  in  online  context  transfer  many  of  leaming-to-leam  approaches  from  traditional  learning  contexts  to  online 
setting.  At  the  same  time,  however,  they  identified  that  online  environment  requests  learners  idiosyncratic  learning 
strategies.  Burge  (1993)  found  learners  using  learning  strategies  with  respect  to  making  choices,  expression,  group 
interaction,  and  the  organization  of  information. 

Eastmond  (1993)  found  that  learners  in  computer  conferencing  based  online  courses  employ  strategies  to 
deal  with  multiple  discussions,  information  overload,  synchronicity,  textual  ambiguity,  and  processing  online 
information  and  determining  what  contributions  to  make.  Conferencing  systems,  basically  with  a hypertext 
structure  and  led  by  learner  participation,  demand  of  learners  a high  level  of  self-regulation  and  meta-cognitive 
abilities  in  the  learning  process. 

Lyman  (1998)  suggests  that  cyber  learners  need  to  develop  learning  strategies  called  ‘information  literacy’ 
in  order  to  take  benefits  from  the  Internet.  The  abilities  especially  required  in  a resource-based  learning 
environment  through  the  Internet  include  the  followings: 

1.  know  when  there  is  a need  for  information 

2.  identify  information  needed  to  address  a given  problem  or  issue 

3.  locate  the  needed  information 

4.  evaluate  the  information 

5.  organize  the  information 

6.  use  the  information  effectively  to  address  the  problem  or  issue 

Cyber  learning  space  is  typically  structured  with  hypertext  and  is  heavily  expected  to  be  the  one  that  is 
strongly  learner  control  oriented.  Learner-oriented  environment  demands  learners  to  make  decisions  by  themselves 
throughout  the  learning  process  and,  therefore,  requires  a high  level  of  self-regulation  and  metacognitive  abilities.  It 
is  important  for  learners  and  in  particular  for  novices  in  a cyber-hypertext  environment  to  acquire  learning  strategies. 
Learners  who  cannot  use  effectively  the  complexity  of  cyber  communication  environment,  therefore,  need  explicit 
modeling  and  scaffolding  support  as  well  as  more  experiences  in  hypertext-based,  open,  flexible  technologies  for 
constructive  learning. 

Gender  issues  in  cyberspace 

Issues  of  relationship  between  technology  and  gender  in  education  have  been  increasingly  explored  in 
recent  years  (Kerr,  1996).  Probably  due  to  the  short  history  of  cyberspace  within  educational  sectors,  however,  little 
attention  has  been  paid  to  the  issues  of  gender  difference  in  and  even  less  regarding  the  issues  of  learning  strategies. 
Most  frequently  appearing  research  issues  are  the  ones  regarding  styles  or  purposes  of  social  interactions  and 
dynamics,  attitudes;  and  styles  and  frequencies  of  expression,  discussion,  and  feedback. 

Methods 

The  subjects  were  156  undergraduate  students,  35  males  and  121  females,  from  a medium  size  university  in 
Seoul,  Korea.  As  a reference,  Korea  is  one  of  the  countries  most  aggressively  experimenting  to  maximize 
educational  benefits  of  the  Internet.  Recent  statistics  indicated  that  Korea  was  the  10th  in  the  Internet  users  as  of  the 
end  of  1999. 

These  students  were  varied  in  terms  of  their  academic  backgrounds  and  years  in  their  programs.  All  of  the 
courses  were  integrated  with  a web-based  instruction  tool,  UniverCampus.  Cyber  learning  and  instruction  in  this 
research  is  basically  asynchronous,  text-based,  open,  and  learning  task  oriented  features;  and  was  operated  with 
combined  with  face-to-face  class.  Instruction/leaming  room  (figure  1)  and  discussion  room  (figure  2)  were  used  as 
two  main  learning  spaces.  Instruction  and  learning  room  provided  learning  materials,  simple  discussion  issues,  and 
quiz  activities.  In  the  discussion  room,  typically  the  professor  posted  discussion  topics  and  specified  the  timeline; 
and  learners  were  expected  to  discuss  about  it  within  the  timeline. 
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Figure  1.  Instruction/Learning  Room  Sample  Screen 


Figure  2.  Discussion  Room  Sample  Screen 


Cyberspace  in  this  research  was  typically  used  to  facilitate  learner-learner  interaction,  more  specifically 
peer  discussion,  idea-sharing,  critique,  feedback,  and  review.  Further,  it  intended  to  facilitate  learner- instructor 
interaction,  including  assignment  posting  and  submission,  and  question-and-answer  session.  In  addition,  it 
supported  to  provide  additional  learning  materials,  as  learner-material  interaction.  Throughout  the  semester, 
learners  were  encouraged  to  contact  assistants  and  the  professor  as  they  need  help.  At  the  outset  of  the  semester, 
one-hour  orientation  covering  how  to  use  the  online  tool  was  offered. 

A 28-item  questionnaire  survey  was  conducted  during  the  final  week  of  the  semester.  The  questionnaires 
were  organized  into  four  categories:  expression,  information  processing,  self-control,  and  the  use  of  human 
resources.  Each  item  was  designed  with  7-point  Likert  type  scales,  using  values  of  1 for  “strongly  agree”  and  7 for 
“strongly  disagree”.  Post-hoc,  internal  consistency  reliability  of  the  survey  was  measured  to  be  Cronbach  r = 86.  T 
test  was  computed  for  the  questionnaires  and  7-point  Likert  like  items  with  negative  descriptions  were  reversed  for 
convenient  interpretation. 
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Results  and  discussion 


Expression 

Responses,  in  general,  indicate  that  males  appear  to  use  more  proactive  and  aggressive  expressions  than 
females  and  feel  less  mental  pressure.  More  males  perceived  themselves  actively  presenting  opinions  (p<0.05)  and 
felt  fewer  difficulties  in  expressing  ideas  in  a written  form  within  the  cyberspace  (p<0.05).  Moreover,  more  males 
are  positive  toward  cyber  discussion  (means  = 3.86);  on  the  contrary,  females  are  more  likely  to  be  negative  toward 
it  (means=4.35)  (p<0.05).  Those  results  can  be  discussed  in  terms  of  core  nature  in  this  research  cyberspace  of  text- 
based,  open,  formal,  and  task-oriented.  The  cyberspace  of  the  present  study  is  text-based  and  open  space,  where 
participants  were  expected  to  conduct  open  discussion  within  specific  learning  purposes,  not  for  personal  and  social 
and  infonnation  dialogues:  Therefore  this  implies  very  much  male  superior  and  male  dominating  features  (Ebben, 
1994;  Fishman,  1997;  Hatton,  1995;  Herring,  1993;  Herring,  1994;  Herring,  Johnson,  & Dibenedetto,  1992; 
Kramarae  & Taylor,  1991;  McDowell  & Schuelke,  1998;  Selfe  & Meyer,  1991;  Spender,  1995;  Tannen,  1991).  In 
this  context,  female  students  tend  to  have  relatively  higher  appreciation  to  communication  than  males; 
motivationally  or  behaviorally  discouraged  eventually  in  posting  own  opinions  or  providing  feedback  to  others. 
These  apprehension  and  dispirit  can  function  against  full  or  appropriate  use  of  learning  strategies  required  to  be 
successful  in  the  cyber  communication  system. 

In  addition,  females  showed  stronger  appreciation  about  expressing  ideas  in  a written  form  in  the  “rear* 
space  (p<0.05).  Those  findings,  as  a whole,  confirm  various  existing  research  that  learner  characteristics  in  real 
space  are  highly  transferred  into  cyberspace  (McDowell,  E.  E.  & Schuelke,  L.  D.,  1998;  Fishman,  1997). 

Processing  of  information 

12  questionnaires  were  asked  with  respect  to  information  overload,  information  decoding,  and  mental 
pressure  of  asynchronous  interaction.  Males  appear  to  outperform  females  in  perceptual  and  behavioral  strategies  as 
well.  If  overloaded,  more  females  tended  to  skip  reading  others’  postings  than  males  (p<0.05).  In  addition,  when 
messages  were  overloaded,  females  (mean  score  = 4.33)  tended  to  feel  more  difficulties  in  selecting  and 
appreciating  them  than  males  (mean  score=3.8)  (t=-1.917,  p =.  057).  ).  These  results  imply  that  more  females  may 
lack  skills  and  strategies  for  managing  information  overload  in  cyber  space  and,  as  a result,  experience  more 
obstacles  in  cyber  learning  process  than  males. 

In  spite  of  no  significant  differences  in  the  degree  of  “reading”  messages  posted  by  other  learners,  the 
degree  of  “dislike”  of  reading  them  was  significantly  different  between  genders  (p<0.05).  This  result  may  indicate 
that  cyberspace  which  is  time  consuming  due  to  its  text  ambiguity  (Mason,  1994)  still  put  more  mental  stress  on 
males,  comparing  to  males. 

On  the  contrary  and  interestingly,  females  outperformed  males  in  submitting  assignments  or  learning 
activities  corresponding  to  instructors’  statement  (p  < 0.01).  There  was  no  statistical  difference  with  respect  to 
motivational  strategy  needed  for  asynchronous  communication  (p=.  066).  Considering  mean  scores,  however, 
males  tended  to  be  more  positive  (means=3.83)  toward  asynchronous  communication;  on  the  contrary,  females  were 
more  likely  negative  (means=4.27). 

Self-control 

Since  cyber  learning  environment  strongly  requires  learner  control  (I-S  Lee,  1999b;  J-K,  Lee,  1999;  J-H, 
Lim,  1999;  M-H,  Jo,  1999;  I-S,  Jung,  1999;  0,  Cho,  1999;  Keegan,  1990;  Mason,  1994;  Bonk  & Dennen,  1999), 
learners  need  self-control  strategies  for  a successful  cyber  learning.  Therefore,  this  research  tried  to  determine 
gender  differences  in  self-control  through  the  questionnaires  employing  the  items  of  time  management,  diligence, 
and  persistence.  The  analysis,  in  general,  revealed  no  statistically  significant  differences  between  genders  except  the 
following.  Females  are  more  likely  to  outperform  males  in  submitting  assignments  or  participating  in  activities 
within  due  date  (p  < 0.05). 

Use  of  human  resources 

Two  items  were  employed  to  determine  whether  any  differences  existed  between  gender  with  respect  to  use 
of  instructors,  assistants,  or  colleagues  in  order  to  solve  any  instruction  or  supportive  problems  during  the  cyber 
learning  process.  The  analysis  revealed  no  statisticallysignificant  differences. 
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Conclusions 

This  research  reveals  significant  gender  differences  in  the  categories  of  expression  strategy  and  information 
processing  strategy,  in  which  males  showed  stronger  abilities  and  positive  attitudes  without  exception.  The  findings 
are  not  entirely  surprising,  since  they  replicate  many  of  the  existing  findings  from  the  areas  of  communication, 
linguistics,  and  sociology  and  more. 

Currently  dominant  modes  of  cyber  courses  demonstrate  structures  and  functions  in  favor  of  males. 
However,  it  should  not  be  taken  for  granted;  we  have  to  move  toward  a gender-equal  cyber  learning  environment. 
This  can  be  achieved  by  including  instructional  design  and  implementation,  which  might  more  likely  respond  to 
learning  strategies  in  favor  of  females.  Of  greatest  potential  leading  to  gender  inequality  in  a cyber  learning 
environment  is  its  text-based,  public,  and  information  overload  natures.  I suggest  the  following  instructional 
interventions  to  overcome  potential  negative  impacts  of  those  natures  on  female  learners. 

First,  considering  that  females  tend  to  experience  difficulties  in  thinking  and  expressing  in  a written  form, 
there  is  a need  for  providing  additional  interaction  modes,  which  might  support  rather  a dialogue-like  interaction. 
Real  time  chatting  is  one  of  the  exemplars  of  instructional  intervention.  In  addition,  sound  materials  and  interaction 
through  voice  messages  should  be  used  more  substantially. 

Second,  considering  that  females  tend  to  experience  difficulties  in  public  postings  and  arguments,  there  is  a 
need  for  learning  opportunities  through  rather  informal  and  social  interactions.  Among  others,  I suggest  designing 
learning  spaces  for  small  group  discussion  and  reading  materials  for  individualized  learning.  Individual  learning 
activities,  rather  than  too  much  focusing  on  learner-learner  interactions,  might  be  instructional  interventions  useful 
for  females  who  demonstrate  rather  high  apprehension  in  public  interaction. 

Third,  considering  that  females  rather  tend  to  experience  difficulties  in  reading  others'  messages,  when 
especially  overloaded,  there  is  a need  for  interface  design  which  might  help  individual  learners  to  search,  organize 
and  present  information  in  their  own  convenient  way.  Concurrently,  it  is  equally  useful  for  mediators  to  regularly 
provide  summary  notes  of  shared  information  and  ideas  among  learners. 
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Fostering  Design  Culture  Through  Cultivating  the  User-Designers’  Design 

Thinking  and  Systems  Thinking 


Li,  Ming-fen 

National  Taiwan  Normal  University 

Cultivating  design  thinking  and  systems  thinking  for  building  a design  culture 

Since  Cross  (1990)  claimed  that  design  is  a unique  mode  of  culture,  like  science  culture  and  humanities 
culture,  the  cultural  aspect  of  design  has  received  more  and  more  attention  in  the  design  community.  Banathy 
(1996)  is  probably  the  most  well-known  scholar  who  profoundly  interprets  Cross’  thinking  about  design  culture.  He 
states  that  it  is  important  to  build  a design  culture  through  cultivating  the  general  public’s  design  competence  and 
design  literacy.  However,  I contend  that  we  need  to  focus  on  ways  of  cultivating  design  thinking  and  systems 
thinking  if  we  want  to  foster  a design  culture  in  the  learning  society.  Because  our  thinking  is  constantly  interacting 
with  our  action,  which  becomes  the  essence  of  our  daily  experience,  and  gradually  shapes  the  forms  and  content  of 
our  culture. 

Indeed,  the  significance  of  design  thinking  and  systems  thinking  to  design  has  gained  much  recognition  in 
recent  years.  Tripp  (1991),  Rowland  (1994),  Akin  (1994),  Nelson  (1994),  Banathy  (1996)  have  expended  much 
efforts  in  analyzing  and  synthesizing  design  thinking  and  systems  thinking  since  1990.  Compared  to  design 
thinking,  scholarly  discussion  about  systems  thinking  has  a much  longer  history  and  includes  more  multiple  and 
divergent  perspectives.  Some  major  systems  theories  have  been  successfully  applied  in  organizational  change,  such 
as  Senge’s  systems  thinking,  Checklend’s  soft  systems  methodology,  AckofFs  design  of  idealized  systems,  and 
Nadler’s  planning  and  design  approach.  However,  in  this  paper,  I would  focus  on  Banathy’s  and  Senge’s  systems 
thinking.  Meanwhile,  I would  propose  an  enlightening  design  approach,  the  DESIGN-WITHIN  approach.  This 
approach  aims  at  enhancing  both  learners’  inner  revolution  and  societal  learning  revolution.  Through  design-within, 
we  could  engage  learners  and  the  learning  community  in  envisioning  their  learning,  and  in  systemically  designing 
their  own  learning.  Indeed,  design  thinking,  as  well  as  contemporary  systems  thinking  are  two  wings  to  make  us  fly 
in  the  spacious  learning  world.  The  design-within  approach  is  an  alternative  approach  which  synthesizes  the  truly 
systemic  spirit  of  systems  thinking  and  the  enlightening  spirit  of  transformative  design. 

The  emergence  of  design  thinking  and  its  multiple  perspectives 

Design  thinking  is  a historically  well-developed  discipline  across  many  fields.  However,  in  the  field  of 
instructional  design,  it  has  long  been  confined  within  strategic  or  systematic  mode  of  thinking.  Its  rich  essence  is  yet 
to  be  discovered.  However,  the  following  contemporary  perspectives  on  design  thinking  have  pointed  out  a much 
spacious  world  for  instructional  designers  to  explore. 

Creativity  and  design 

While  clarifying  the  essence  of  instructional  design  and  development,  Davies  (1981,  1994)  attempts  to 
uncover  the  multiple  facets  of  it.  The  way  he  frames  design  in  the  craft,  science,  and  art  facets  is  very  illuminating. 
He  pinpoints  the  kinds  of  design  and  development  possibilities  from  the  “process”  perspectives,  and  proposes  a 
“process  design”  approach  for  corporate  training.  Rowland  and  Wilson  (1994)  look  into  the  creative  processes  of 
design  by  exploring  the  essence  of  Csikszentmihalyi’s  flow  psychology.  They  contend  that  to  situate  the  designers 
in  liminal  states  is  critical  to  nurture  their  creative  thinking.  They  enumerate  several  contexts  that  could  trigger  the 
designers’  mental  occurrence  of  liminal  states.  Most  important,  they  depict  design  as  the  designerly  ways  of 
knowing,  which  is  resonant  with  Banathy’s  interpreting  design  as  a mode  of  inquiry.  Meanwhile,  Rowland  (1993) 
investigates  the  design  process  of  expert  designers  across  many  different  design  disciplines.  By  synthesizing  the 
commonalities  underpinning  many  design-related  fields,  he  enriches  our  thinking  about  and  understanding  of  the 
“design  process”. 

Compatively  speaking,  the  way  Nelson  (1994)  approaches  design  is  quite  unique.  He  contends  that  design 
as  learning  is  quite  different  from  the  traditional  understanding  of  learning.  He  uses  the  concept  of  feedback  for 
further  explanation.  He  argues  that  “feedback  in  learning  process  is  typically  presented  as  positive  or  negative. 
Design  learning  involves  another  less  known  system’s  process  that  can  best  be  characterized  as  feedforward;  inquiry 
pulled  by  volition  and  purpose.  It  is  not  about  discovering  correct  or  incorrect  predictions  neither  about  describing 
reality  in  terms  of  truth.  It  is  discovering  knowledge  at  the  edge  between  chaos  and  order;  between  the  unknown 
and  the  known.  Design  learning  is  also  pragmatic  in  the  discovery  and  mastery  of  skills  needed  to  bring  about 
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abstract  concepts  into  lived  reality  (p.52).”  Nelson  (1994)  advocates  that  we  should  have  more  patience  to  live  with 
fuzziness  of  design  and  to  undisciplined  design.  He  also  claims  that  design  includes,  but  goes  beyond,  the  theory  of 
a science  of  the  artificial  in  that  it  deals  with  the  social  organizations,  patterns  of  human  interactions,  and  functional 
social-technical  structures  that  serve  human  purposes.  Design  is  a synthesis  of  creativity  (imagining  new  things)  and 
innovation  (bringing  those  new  things  into  existence)  within  this  multi-dimensional  domain.  It  is  both  process  and 
artifact.  Compared  to  Davies  and  Rowland,  Nelson  seems  to  hold  a more  moderate  stance. 

Design  as  problem-solving  vs  problem-restructuring  vs  dialogical  process 

As  an  architecture  design  researcher,  Akin  has  very  different  perspectives  on  problem  solving  and  problem 
restructuring  from  many  other  researchers.  He  affirms  that  problem-restructuring  is  by  no  means  a newly 
discovered  phenomenon,  but  has  a rather  long  history  in  design  research  and  even  historical  discover  on  reasoning. 
We  need  to  be  aware  that  here  Akin  focuses  much  on  “problem  restructuring”,  rather  than  on  problem-solving  or 
problem  framing.  It  is  such  problem-restructuring  process  that  underlies  design,  and  represents  design  as  a unique 
profound  discipline.  We  can  say  that  the  way  Rowland  situates  designers  in  the  liminal  states  for  generating  creative 
design  is  very  similar  to  the  way  Akin  emphasizes  on  engaging  designers  in  the  problem-  restructuring  process. 
Beyond  our  common-sense  knowledge  of  problem-solving,  Akin  holds  the  views  that  the  less  understood  a problem 
domain,  the  more  the  degrees  of  freedom  the  designer  has  and  less  reliance  on  standard  solutions  can  be  expected. 
Like  Rowland  (1994)  and  Banathy  (1994)  who  believe  that  to  design  is  to  leap  out  from  the  present  to  the  possible 
future,  Akin  thinks  the  process  of  creative  design  is  to  leap  out  of  the  framework  to  explore  other  possibilities  or 
expand  the  boundary  of  the  problem  domain.  Although  Schon’s  research  on  professional  practice  is  not  limited  to 
design-related  fields,  his  thinking  about  problem  solving  has  great  implication  on  design.  In  fact,  Schon  (1985)  is 
more  pragmatic  in  terms  that  he  values  both  problem-solving  and  problem-restructuring  process.  He  regards 
problem  solving  and  problem  restructuring  as  the  indispensable  dual  process,  which  provide  us  with  the  moderate 
way  of  knowing  and  action  about  design. 

From  these  discussions,  we  could  find  that  design  thinking  encompasses  multiple  perspectives.  Instead  of 
listing  the  various  definitions  of  design,  I would  first  synthesize  Tripp’s  (1991)  and  Rowland’s  (1993)  discussions 
about  the  design  process  and  design  nature,  and  then  summarize  Banathy’s  discussion  about  design  nature  and 
design  culture  in  the  following  section.  Tripp  (1991)  reviews  numerous  empirical  studies  of  design  across  the 
general  design  field,  such  as  engineering,  architecture,  software  design.  In  probing  the  nature  of  design,  Tripp  tries 
to  clarify  if  a theory  of  design  can  be  constructed.  Thus,  he  synthesized  many  divergent  views  of  design  into  two 
general  theories  of  design.  According  to  Tripp,  there  are  two  different  theories  of  design,  namely  design  as 
optimization  and  design  as  dialogue.  The  former  is  in  the  same  vein  of  Simon’s  perspectives  (1981)  whereas  the 
latter  is  in  aligning  with  Schon’s  (1987)  reflection-in-action.  They  have  various  focuses.  Design  as  optimization 
usually  takes  logical,  rational,  systematic  processes.  Yet,  design  as  dialogue  takes  intuitive,  creative,  artistic 
processes.  In  terms  of  problem-solving  in  the  design  process,  optimized  design  places  emphasis  on  complete 
understanding  prior  to  solution  attempts  in  order  to  finding  the  best  description  of  the  problem.  However,  dialogical 
design  takes  early  attempts  at  solution.  Therefore,  it  is  not  like  optimization  design  that  treats  design  as  an  instance 
of  problem  solving.  Instead,  design  is  treated  as  a reflection-in-  action  process,  and  a social  process  of  negotiation, 
hermeneutic  in  nature.  In  contrasting  to  optimization  design’s  focus  on  formal  representation  of  problem-heuristics 
activities,  dialogical  design  usually  embeds  uncertainty,  uniqueness  and  conflict. 

From  the  above  discussion,  we  could  find  that  design  has  been  inquired  from  various  perspectives,  which 
indeed  range  from  design  product  (artifact),  to  design  skills  and  strategies,  to  design  process,  and  to  design  thinking. 
Rowland  traces  the  process  of  design  by  finding  out  the  converging  arena  of  instructional  design  with  design.  He 
studies  the  relationships  between  design  and  instructional  design  from  the  4 facets:  the  purposes  or  goal  of  design, 
relationships  to  other  processes,  factors  that  influence  the  design  process,  and  the  nature  of  design  process.  In 
addition  to  exploring  the  designers’  creativity,  and  the  design  expertise,  Rowland  extends  much  of  his  scholarly 
discussion  to  the  issue  of  nurturing  the  dialogical  culture  within  the  design  community.  This  is  very  different  from 
simply  embedding  the  dialogical  strategy  to  the  design  process.  Indeed,  Rowland  and  Wilson’s  conceptions  of 
designerly  ways  of  knowing  and  Rowland’s  discussions  about  design  “culture”  make  design  thinking  the 
indispensable  knowledge  base  of  instructional  design. 

Design  culture 

While  clarifying  the  nature  of  design,  Banathy  (1996)  enumerates  the  various  definitions  of  design  from 
various  design-related  disciplines.  He  found  that  the  various  definitions  of  design  convey  the  notion  that  design  is 
practiced  by  many  professions,  in  many  different  ways,  and  is  applied  in  various  contexts.  When  extending  Cross’ 
design  as  a mode  of  culture,  he  advocates  that  we  should  not  leave  design  decisions  affecting  our  society  to  the  so- 
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called  design  experts.  Instead,  we  should  include  a broad-based  participation  of  the  users  in  the  design  of  their 
systems.  We  should  build  design  cultures  that  include  the  general  public  in  order  to  complement  the  expert  culture 
of  professional  designers.  However,  if  we  want  to  include  the  general  public  in  the  design  process,  we  need  to 
cultivate  the  public’s  literacy  in  design.  He  affirmatively  states  that  our  era  can  surely  be  called  the  age  of  design. 
He  suggests  that  in  the  age  of  design,  the  building  of  a design  culture  is  an  inescapable  necessity.  Because,  if  we 
take  away  design,  we  strip  the  world  of  most  of  its  enabling  mechanism.  He  urges  the  public  to  make  choice  for 
themselves,  not  simply  relying  on  the  experts’  design  thinking  and  decisions.  Because  we  can  either  live  with  the 
poor  design  by  the  so-called  experts,  or  we  can  empower  ourselves  by  acquiring  design  literacy  and  design 
competence  so  that  we  can  assume  responsibility  for  the  design  of  systems  in  which  we  live  and  to  which  we  are 
connected. 

Indeed,  we  should  not  be  engaged  in  design  simply  to  fix  or  complemement  certain  aspects  of  the  whole 
learning  environment.  What  we  need  is  to  create  a new  image  which  is  more  encompassing  than  the  original  one 
through  reflective,  critical  and  enlightened  thinking.  We  also  need  to  cultivate  the  designers’,  user-designers’  or  the 
design  community’s  critical  awareness  of  their  beliefs  and  values  that  underlying  their  design  thinking  and  action. 
Like  the  way  Senge  clarifies  unwhished  vision  and  wanted  vision,  Banathy  makes  clear  difference  between 
improvement  and  restructuring.  Through  such  clarification  and  distinguishment,  we  could  better  understand  that 
design  is  a unique  mode  of  thinking,  not  an  add-on  to  any  other  disciplines.  In  particular,  Page’s  (1966)  “design  as 
an  imaginative  jump  from  present  facts  to  future  possibilities”  seems  to  provide  a clear  direction  for  designers  to 
strive  for.  But,  here  emerge  some  fundamental  questions: 

• How  could  we  transform  the  current  reality  by  design? 

• Do  we  need  any  other  disciplines  than  design  to  attain  such  transformation? 

• While  trying  to  generate  the  design  image  of  the  new  reality,  how  should  we  cope  with  our  own  habitual 
design  thinking  and  action,  which  is  so  deeply  ingrained  in  our  mental  models? 

• How  would  users  and  designers  interact  in  the  design  culture? 

Link  between  design  thinking  and  systems  thinking 

Although  design  thinking  and  systems  thinking  seem  to  be  literally  different,  they  are  two  interrelated 
concepts  when  applied  in  learning  or  instructional  design.  When  dealing  with  design  issues,  instructional  designers 
inevitably  face  the  complicated  learning  or  educational  systems.  Indeed,  design  is  a powerful  bridge  to  integrate 
systems  thinking  into  learning  or  instructional  practice.  This  is  why  Banathy  (1996)  highlights  systems  thinking  as 
the  conceptual  environment  or  parent  of  design  thinking.  He  regards  that  design  is  one  of  several  disciplined  inquiry 
domains  of  social  systems  in  which  systems  thinking  is  manifested. 

Banathy ’s  intellectual  technology — “designing”  social  “systems” 

Indeed,  Banathy  has  been  engaging  himself  in  the  design  inquiry  since  the  1960s.  While  he  is  well  known 
as  an  enthusiast  for  systemic  design  of  educational  systems,  his  efforts  in  advocating  design  literacy  and  culture  is 
less  well  known.  Part  of  the  reason  may  be  because  of  the  complexity  and  profundity  of  his  design  thinking.  In 
“Designing  social  systems  in  a changing  world”,  Banathy  (1996)  endeavors  in  historical  and  extensive  scholarly 
discussion  on  design.  He  compares  design  and  other  modes  of  inquiry,  and  treats  design  as  one  of  the  three  cultures, 
science,  humanities  and  design.  Furthermore,  he  tries  to  envision  the  new  social  systems  by  regenerating  design, 
trying  to  bring  design  to  the  social  system,  the  educational  contexts,  and  everyone’s  life.  His  design  thinking  is 
closely  linked  to  his  systems  thinking.  In  another  words,  his  design  thinking  is  to  be  realized  and  carried  out  in  the 
open  social  systems.  We  can  say  that  he  has  depicted  a spacious  and  magnificent  landscape  for  the  designers  to 
travel. 

Banathy  advocates  that  we  need  creative  and  proactive  design  visions  to  transform  the  social,  educational 
reality.  He  not  only  defines  design  and  social  systems,  but  also  comprehensively  compares  design  and  other  modes 
of  inquiry,  and  approaches  design  from  the  cultural  visions.  He  defines  design  as  a multidimensional  inquiry  by 
synthesizing  systems  thinking,  design  thinking  and  other  multiple  perspectives.  He  also  tackles  the  ethics  of  social 
system  design  and  the  design  of  the  ideal  system.  In  essence,  he  integrates  a wide  range  of  knowledge  in 
approaching  his  inquiry  of  design.  His  contribution  to  design  inquiry  is  that  he  elegantly  and  profoundly  synthesizes 
design  thinking  and  systems  thinking. 

Nelson’s  designers  as  symbolic  synthesizers 

While  Banathy  conceptualizes  social  systems  design  as  intellectual  technology,  and  integrate  systems 
thinking  in  his  social  systems  design  inquiry,  Nelson  cautiously  probes  the  reasons  for  engaging  in  systems  design. 
He  claims  that  we  need  to  know  about  why  systems  design  is  in  need  for  the  new  age,  and  contemplates  that  system 
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designers  as  symbolic  “synthesists”,  whose  role  is  inclusive  of  what  a symbolic  analyst  does,  is  an  emerging 
significant  role  for  a complex  world,  working  in  environments  of  complexity. 

He  asserts  that  new  times  need  new  designs.  Therefore,  learning  systems  design  should  involve  the 
synthesis  of  two  very  important  intellectual  traditions  which  are  gaining  prominence  with  the  establishment  of  the 
information  age:  systems  thinking  and  design  action.  He  further  explains  that  systems  thinking  provides  a 

framework  for  describing  or  conceptualizing  the  complexity,  interconnectedness,  and  nonlinear  dynamics  of 
institutional  and  organizational  systems,  while  design  provides  the  action  framework  for  how  to  visualize  and  bring 
into  existence,  in  functional  form,  telelogic  systems  (i.e.,  serving  human  purpose).  Of  the  two,  systems  thinking  is 
the  most  developed  theoretically.  Design  is  the  most  developed  pragmatically  but  is  in  need  of  the  most 
development  conceptually  in  order  to  more  fully  enhance  the  synergistic  potential  in  combination  with  existing 
systems  thinking. 

Nelson’s  (1994)  thinking  about  design  is  special  in  three  aspects:  First,  he  argues  that  problem  solving  and 
design  are  entirely  two  different  phenomenons.  Design  is  a special  way  of  thinking-different  from  scientific 
thinking-that  is  performed  ’’out  of  control”,  with  a deliberate  break  from  restrictions  on  imagination.  Second,  in 
defining  design,  he  distinguishes  between  self-expression  of  the  arts  and  other  expression  of  design.  He  describes 
the  relationship  between  designer  (self)  and  the  client  (other)  as  synergistic,  as  becoming  more  capable  of  more  than 
the  client  or  designer  separately.  Third,  like  Banathy,  he  links  systems  thinking  to  design  thinking,  and  argues  that 
we  need  to  move  away  from  thinking  in  separate  disciplines  of  expertise.  He  offers  the  integration  of  systems  and 
design  as  a powerful  perspective  for  meeting  current  organizational  challenges. 

Systems  thinking  as  another  wing  to  take  us  to  the  spacious  learning  world 

Systems  thinking  has  played  a critical  role  in  the  historical  development  of  educational  technology  since 
1970s.  The  instructional  design  models  which  adopt  systems  thinking  were  once  widely  recognized  in  many 
educational  and  training  contexts.  Despite  the  positive  recognition  of  its  value  to  education  and  training,  the  essence 
systems  thinking  is  narrowed  down  to  “systematic”  view,  distorted  its  true  spirit  greatly.  While  constructive 
learning  and  situated  learning  gain  more  and  more  attention  in  the  area  of  instructional  design,  the  way  systems 
thinking  has  been  interpreted  and  applied  is  also  challenged.  Although  some  scholars  turn  to  systemic  views  to 
broaden  their  perspectives  on  systems  thinking,  its  essence  has  yet  been  thoroughly  studied.  I believe  that  deeper 
understanding  and  discussion  about  various  modes  of  systems  thinking  is  essential  for  uncovering  the  potential 
significance  of  systems  thinking  in  terms  of  fostering  design  culture.  At  the  fundamental  level,  cultivating  user- 
designers’  systems  thinking  for  design  is  more  important  than  developing  more  systematic  or  systemic  models  for 
them  to  use. 

Therefore,  in  addition  to  reviewing,  analyzing  and  comparing  the  design  thinking  mentioned  above,  I 
would  focus  on  Banathy ’s  and  Senge’s  systems  thinking.  Because  Banathy  not  only  synthesizes  and  critically 
reviews  many  system  philosophers’  systems  thinking,  but  also  integrates  successfully  systems  thinking  into  his 
design  inquiry.  He  integrates  a wide  range  of  knowledge  of  design  thinking  and  systems  thinking  when  undertaking 
his  design  inquiry  to  transform  the  social  systems.  His  contribution  to  design  inquiry  is  that  he  elegantly  and 
profoundly  synthesizes  design  thinking  and  systems  thinking  by  pointing  out  the  route  to  transforming  the  existing 
educational  systems  into  the  ideal  creative  system.  Senge’s  systems  thinking,  as  he  refers  to  as  the  fifth  discipline,  is 
embodied  in  and  interacting  with  other  disciplines  (i.e.  personal  mastery,  shared  vision,  team  learning  & mental 
model)  to  help  a team  or  organization  to  cultivate  their  learning  culture.  His  systems  thinking  and  other  disciplines 
to  be  practiced  with  systems  thinking  has  been  widely  recognized  in  restructuring  organizational  learning  culture.  In 
the  past  two  years,  he  and  several  other  researchers  and  practitioners  also  develop  organizational  learning  through 
the  five  disciplines  in  the  educational  contexts.  Therefore,  his  systems  thinking  as  well  as  other  disciplines  has  great 
implications  on  fostering  an  organization’s  learning  culture. 

Systems  thinking  and  open  educational  systems  design 

Banathy’s  visions  of  systems  thinking  underwent  great  change  in  the  1970s.  Earlier  than  that,  he  regarded 
instructional  systems  as  close  systems  to  be  well  thought  and  designed  like  many  instructional  technologists.  In  the 
book  “Instructional  Systems”,  he  was  still  confined  within  closed  systems  engineering  type  of  thinking  to  construct 
the  instructional  systems.  Even  so,  the  instructional  systems  he  proposed  is  much  more  systemic  compared  to  many 
instructional  design  and  development  models  in  the  1970s  and  1980s.  Yet  he  believes  that  as  long  as  the 
instructional  systems  are  closed,  they  are  useless  in  the  domain  of  social  systems,  and  even  counter  the  spirit  of 
learning  and  education.  Therefore,  he  proposes  an  open-systems  view  to  deal  with  the  educational  problems,  and 
published  “Developing  a systems  view  of  education:  A systems  models  approach  in  1973.  In  the  past  2 decades,  he 
has  been  advocating  his  design  thinking  and  systems  thinking  to  designing  social  systems.  He  attempts  to  diffuse 
systems  thinking  and  the  dynamism  of  it  to  the  educational  community.  In  1996,  he  finished  his  system  design 
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theory,  which  encompasses  3 models  for  redesign  the  educational  system,  i.e.  a system-environment  model,  a 
functions/structure  model  and  a process  model.  The  three  models  are  to  be  applied  in  two  aspects:  First,  they  are  to 
depict  the  systems  concepts  and  the  systemic  level  of  the  way  we  understand  the  educational  structure  of  the  existing 
systems,  and  the  way  we  evaluate  the  educational  environment.  Second,  they  are  used  to  design  the  new  educational 
systems  by  comparing  the  gap  between  the  ideal  conceptual  systems  and  the  existing  systems.  He  not  only  designed 
the  3 models  but  also  provide  many  activates  to  guide  the  users  to  apply  his  models,  through  which  users  might 
gradually  cultivate  their  systems  thinking  and  systems  application  capabilities.  In  terms  of  this,  Banathy  seems  to  be 
much  closer  to  Senge,  both  of  whom  do  not  intend  to  merely  invent  models  and  theories  for  users  and  learners  to 
apply,  but  engage  them  in  the  thinking  process  and  problem-solving  processes.  Unlike  many  instructional 
technologist  who  make  much  efforts  on  constructing  design  models  and  theories,  Banathy  claims  that  more  efforts 
should  be  expended  to  cultivate  the  instructional  technologists  systems  thinking,  and  their  literacy  of  systems 
theories  and  systems  methodology,  which  he  has  been  persistently  working  on. 

In  referring  to  Stafford  Beer’s  vision  that  human  beings  are  prisoned  by  their  own  thinking,  Banathy  tries  to 
construct  his  systems  models  to  expand  our  cognitive  power,  and  enhance  our  capabilities  to  deal  with  complexity. 
He  found  that  traditional  science  defines  complexity  by  examining  the  multiple  components  within  a system, 
whereas  systems  science  defines  complexity  by  the  interaction  between  the  system  and  its  environment,  and  by  the 
relationship  among  the  components  within  the  system.  What  makes  a difference  is  that  the  former  is  a close,  static 
system,  but  the  latter  is  an  open,  dynamic  system.  It  is  essential  to  recognize  that  Banathy’s  systems  thinking  is  to 
be  applied  in  the  open  social  systems,  and  closely  linked  to  the  concepts  of  synthesis  and  expansion,  rather  than 
analysis,  which  underlies  most  traditional  instructional  design  and  development  models.  He  argues  that  through 
synthesis  and  expansion  we  can  better  understand  the  systems  and  its  relationship  with  the  larger  systems  or 
environments.  Meanwhile,  we  should  shift  from  anticipating,  predicting  and  controlling  the  human  world  to 
understanding  the  uncertainty  and  complexity  of  the  environment. 

However,  while  applying  Banathy’s  systems  models  to  designing  educational  systems,  we  need  to  reflect 
upon  how  our  habitual  thinking,  acting  and  problem-solving  patterns  might  counter  the  new  design  thinking  or 
systems  thinking.  Because  no  matter  how  sounding  the  systems  methodology  or  models  are,  human  thinking  and 
acting  tend  to  follow  their  habitual  routes,  rather  than  new,  less  traveled  routes.  Especially  when  the  user-designers 
are  engaged  in  team  work,  such  reflection  on  the  gap  between  their  old  and  new  thinking  is  even  more  important. 
Because  everyone’s  understanding  and  acting  upon  the  system  methodology  and  models  may  diverge  at  different 
levels  and  dimensions.  Therefore,  how  to  cultivate  all  design  participants  to  use  design  language  or  system  language 
at  a more  communicable  platform  becomes  a fundamental,  significant  issue.  If  users  or  learners  do  not  have  full 
understanding  and  communication  of  the  visions  of  the  ideal  systems,  or  could  not  form  a design  team,  they  might 
add  another  dimension  of  complexity  to  the  system  itself.  Indeed,  the  human  factors  which  interfere  with  or  even 
fail  many  restructuring  tasks  in  education  are  not  uncommon  at  all. 

From  Banathy’s  systems  thinking,  we  have  a much  broader  view  of  what  systems  are  and  how  systems 
thinking  might  be  applied  to  solve  the  existing  problems  or  design  new  educational  systems.  Such  systemic 
perspectives  do  help  expand  our  cognitive  power  about  systems  and  reconceptualize  our  rigid  understanding  of 
instructional  or  educational  systems.  However,  as  mentioned  earlier,  the  user-designers  or  learners  should  be 
provided  with  ample  opportunities  to  reflect  upon  their  habitual  thinking  and  acting  which  might  have  interfered 
with  their  engagment  in  systems  thinking  or  even  prevented  them  from  understanding  the  new  mode  of  thinking. 
Because  transforming  one’s  inner  habitual  thinking,  acting  and  ways  of  design  may  be  more  difficult  than  redefining 
and  clarifying  the  system  problems,  or  generating  strategies  to  create  the  ideal  systems.  To  overcome  this,  we  could 
gain  much  insights  from  Senge’s  systems  thinking. 

If  we  could  undertake  the  task  of  design  from  alternative  perspectives,  we  might  be  able  to  shift  design 
focus  from  designing  learning  product  and  environments  to  cultivating  their  thinking  and  actions  about  design. 
Below,  I will  analyze  and  critique  Senge’s  systems  thinking,  and  its  practice  with  other  disciplines  for  regenerating  a 
learning  organization. 

The  essence  of  Senge’s  systems  thinking:  Philosophical  roots  in  both  western  and  eastern  culture 

Senge’s  systems  thinking  has  profound  philosophical  roots  in  both  western  and  eastern  cultural  traditions. 
While  developing  his  systems  thinking,  he  converges  much  ancient  Chinese  wisdom  with  traditional  western 
philosophies,  and,  furthermore,  integrates  them  into  the  theory  of  systems  dynamics.  If  we  interpret  Senge’s 
systems  thinking  without  recognizing  its  profound  philosophical  roots,  we  may  not  be  able  to  capture  the  wholeness 
of  his  systems  thinking.  Through  critically  reviewing  the  essence  of  Senge’s  systems  thinking,  I found  that  his 
systems  thinking  is  framed  from  views  of  learning,  time  and  space,  cause  and  effect,  universe,  structure,  and 
working  ethics.  He  argues  that  “learning”  has  lost  its  central  meaning  in  contemporary  usage,  because  learning  has 
come  to  be  synonymous  with  “taking  in  information.”  Yet,  taking  in  information  is  only  distantly  related  to  real 
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learning.  Learning  should  involve  a fundamental  shift  or  movement  of  mind.  Real  learning  gets  to  the  heart  of  what 
it  means  to  be  human.  It  is  through  such  learning  that  we  re-create  ourselves.  In  other  words,  true  learning  should 
be  transformation  of  spirit  and  mind,  not  merely  accumulation  of  information  or  knowledge.  Furthermore,  he 
proposes  that  we  need  to  transform  the  technical  mode  of  working  into  the  spiritual  pursuit  of  working  ethics. 

Senge  also  thinks  that  our  distorted  views  of  time  and  space  often  destroy  our  patience  because  we  can 
easily  encounter  a current  problem,  which  is  indeed  resulted  from  yesterday’s  solution.  The  relationship  between 
cause  and  effect  is  not  closely  related  in  time  and  space  as  we  always  imagine  and  expect.  Very  often  our 
impatience  and  intolerance  with  the  effect  makes  us  rush  into  erroneous  judgment  or  decision-making,  which  is  even 
worse  than  no  judgment  or  decision  at  all.  His  perspectives  on  universe  distinguishes  him  from  other  systems 
thinkers  greatly.  He  does  not  define  the  components  of  systems  or  categorize  the  types  of  systems,  but  emphasizes 
that  the  line  between  inside  and  outside,  parts  and  wholes  is  indeed  very  arbitrary.  Because  we  tend  to  treat 
problems  as  if  we  are  outsiders,  rather  than  treating  the  problems  and  ourselves  as  oneness,  we  often  unconsciously 
complicate  the  nature  of  the  problems.  Based  on  these  philosophical  thinking  and  perspectives,  he  develops  several 
laws  of  systems  thinking,  such  as  “Today’s  problems  come  from  yesterday’s  solutions.”,  “The  harder  you  push,  the 
harder  the  system  pushes  back.”,  “The  cure  can  be  worse  than  the  disease.” 

Systems  thinking  as  a thinking  language  to  be  cultivated 

Like  many  other  systems  thinkers,  such  as  Banathy  and  Checklend,  Senge ’s  systems  thinking  is  also  a 
discipline  for  seeing  wholes,  a framework  for  seeing  interrelationships  rather  than  things.  But,  his  systems  thinking 
is  not  only  characterized  by  seeing  wholeness  and  interrelatedness  of  systems,  but  also  , by  seeing  patterns  of 
change.  He  regards  systems  thinking  as  a discipline  for  seeing  the  “structure”  that  underlie  complex  situations,  and 
for  discerning  high  from  low  leverage  change.  Because  he  thinks  structure  influences  behavior,  and  structure  in 
human  systems  is  subtle.  The  reason  that  structural  explanations  are  so  important  is  that  only  they  address  the 
underlying  causes  of  behavior  at  a level  that  patterns  of  behavior  can  be  changes.  Therefore,  he  claims  that  we  need 
to  look  into  the  underlying  structure  of  many  organizational  behaviors,  not  being  confined  within  the  incidents  or 
patterns  of  behaviors.  It  is  only  when  we  see  the  systemic  structure  could  we  be  collectively  engaged  in  generative 
organizational  learning,  not  responsive  or  reactive  learning. 

Another  unique  aspect  of  Senge’s  systems  thinking  is  that  Senge  constructs  systems  thinking  as  a thinking 
language  to  be  learned,  practiced  and  cultivated.  It  is  used  to  capture  the  structures  underlying  the  individuals’  or 
organizations’  behavior  structure.  He  develops  3 primary  thinking  tools  for  cultivating  systems  thinking: 
(l)feedback  loop  1 -reinforcing  loops:  when  small  changes  become  big  changes,  (2)  feedback  loop  2 — balancing 
loops:  pushing  stability,  resistance,  and  limits,  and  (3)delays:  when  things  happen. . ..eventually.  In  order  to  guide  us 
to  learn  these  thinking  tools,  he  develops  four  toolboxes.  They  are  (1)  learning  how  to  draw  systems  maps, 
including  the  interaction  between  cause  and  effect,  its  dynamic  loop,  system  feedback  perspectives,  anti- 
agnosticism, and  sharing  systems  problems,  (2)  learning  how  to  describe  reinforcing  loops,  (3)  learning  how  to 
describe  balancing  loops,  and  (4)  learn  how  to  describe  delays.  Senge  identifies  the  limitation  of  our  written  and 
spoken  language,  and  thus  tries  to  demonstrate  to  us  how  systems  thinking  can  be  represented  through  drawing 
systems  maps,  systems  loops,  nodes  and  time  delay.  He  translates  his  long-term  observation  of  and  experience  with 
recurrent  organizational  learning  development  and  problems  into  “systems  archetypes.”  Such  systems  archetypes 
are  used  to  frame  structural,  and  rooted  problems  of  certain  organizational  behaviors  or  phenomenon.  They  are 
found  in  many  organizations,  include  limits  to  growth,  shifting  the  burden,  eroding  goals,  escalation,  success  to  the 
successful,  tragedy  of  the  commons,  fixes  that  fail,  and  growth  and  underinvestment.  In  addition,  he  regards 
systems  thinking  as  the  fifth  discipline  which  is  the  conceptual  cornerstone  underlying  all  of  the  five  learning 
disciplines,  team  learning,  shared  vision,  mental  model,  personal  mastery  and  systems  thinking  itself.  According  to 
Senge,  all  of  the  five  disciplines  “are  concerned  with  a shift  of  mind  from  seeing  parts  to  seeing  wholes,  from  seeing 
people  as  helpless  reactors  to  seeing  them  as  active  participants  in  shaping  their  reality,  from  reacting  to  the  present 
to  creating  the  future,  (p.69).”  We  should  identify  the  fact  that  systems  thinking  has  been  successfully  integrated  to 
the  teaching  of  many  school  subjects  and  many  organizational  learning-training  programs.  Its  potential  and  value  to 
education  really  needs  further  exploration. 

Comparison  of  Banathy’s  and  Senge’s  systems  thinking 

Generally  speaking,  Senge’s  systems  thinking  not  only  has  Banathy’s  systemic  view,  but  also  has  its  unique 
features.  From  his  profound  systemic  views,  self  and  others  are  inseparable  oneness.  Therefore,  the  purpose  of 
applying  systems  thinking  is  not  to  depict  the  details  of  the  component  systems,  and  the  relationships  among  the 
major  and  minor  systems.  It  is  to  frame  and  reframe  our  own  problems  by  situating  ourselves  within  the  systems 
which  we  intend  to  understand  and  interpret.  Meanwhile,  his  systems  language  translates  complexity  of  reality  into 
simplicity  of  wisdom,  with  which  we  could  better  identify  the  structural  and  rooted  problems  embedded  in  our  own 
thinking  and  action.  Systems  archetypes  become  powerful  tools  for  converting  complicated  problems  or  systems 
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into  simple  communicable  language.  In  addition,  Senge’s  systems  thinking  is  intended  to  be  practiced  with  other 
disciplines,  personal  mastery,  mental  models,  team  learning,  and  shared  visions.  All  of  the  five  disciplines  cannot 
be  cultivated  in  isolation.  Through  the  mutual  cultivation  of  the  five  disciplines,  each  discipline  can  exert  its 
greatest  learning  impact  on  the  team  or  organization.  The  five  disciplines  have  been  widely  recognized  and 
successfully  practiced  in  many  organizations.  It  is  time  for  us  to  think  about  its  value  to  education. 

In  the  past  decades,  educational  technologists’  visions  of  systems  thinking  have  been  narrowed  down  to 
systematic  ways  of  design.  We  should  expand  our  thinking  and  discussion  about  the  true  nature  of  systems  thinking, 
and  allow  more  multiple  perspectives  on  systems  thinking  to  enrich  our  visions.  We  have  focused  too  much  on 
defining  the  outer  educational  systems,  ignoring  the  inner  systems  within  which  most  of  us  are  prisoned.  We  may 
need  to  start  thinking  about  how  to  bridge  our  inner  thinking  systems  with  the  outer  systems  through  Senge’s 
cultivation  of  shared  visions,  team  learning  and  reconstruction  of  mental  models.  It  is  significant  to  situate  all 
learners,  teachers  and  designers  in  the  boundless  circle  of  the  educational  systems.  We  should  also  learn  to  live  with 
the  fuzziness  of  system  boundaries,  which  might  have  seriously  distorted  our  perspectives  of  systems.  If  more  and 
more  designers  could  integrate  the  systems  thinking  proposed  by  Banathy  and  Senge  into  their  practice,  design  will 
no  longer  be  treated  as  models  or  theories  to  be  applied.  When  the  design  community  becomes  more  concerned 
about  ways  of  expanding  design  visions  and  cultivating  design  thinking,  design  culture  could  be  possibly  fostered. 
Below  is  the  comparison  and  synthesis  of  Senge’s  and  Banathy’s  systems  thinking. 

Table  1:  Comparison  of  Banathy ’s  and  Senge ’s  systems  thinking 


The  1st  generation 
Designer 

* enters  a system  as  an  outside  expert 

* creates  an  image  of  the  future  system 
* hands  it  over  to  the  clients  for  implementation 

The  2nd 

generation  designer 

* has  slightly  more  interaction  with  clients 
* asks  clients  for  feedback  on  the  final  draft  before  turning  it  over 
to  them  for  implementation 

The  3rd 

generation  designer 

* enters  an  organization  as  outside  expert 
* draws  up  plans  for  the  design  or  redesign  of  a system 
with  even  more  interaction  with  clients. 

* invites  input  and  feedback  from  the  clients  throughout  the  design  process  and 
even  assist  them  with  the  implementation. 

The  4th 

generation  designer 

* takes  a radically  different  role  in  the  design  process 
* functions  primarily  as  a learning  facilitator  to  help  clients  learn  to  design  fo 
themselves 

* shifts  primary  responsibility  for  learing  and  designing  to  the 

clients 

* shoulders  responsibility  for  fostering  participants’  design 

competence 

* “does  with”  the  clients,  instead  of  “does  to”  or  “does  for”  the  clients 
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Lookforward — the  design-within  approach 

Indeed,  no  matter  whether  design  is  a problem-solving,  problem-reframing,  or  dialogical  process,  we  need 
to  be  aware  if  we  are  doing  design  in  the  mode  of  “reflection  in  action”  or  in  our  original  habitual  thinking,  acting 
and  ways  of  design.  We  also  need  to  cultivate  our  critical  awareness  and  understanding  of  the  essence  of  design 
thinking  and  systems  thinking.  Especially  when  design  in  a team  or  organization,  we  need  more  space  to  allow  one 
another  to  navigate  so  that  design  visions  can  be  shared,  and  one’s  beliefs,  values  and  mental  models  can  be 
transformed  through  team  learning.  It  is  through  the  cultivation  of  design  thinking  and  systems  thinking  in  a 
learning  team  and  organization  that  design  culture  could  gradually  be  fostered.  After  reviewing  and  critiquing  the 
essence  of  design  thinking  and  systems  thinking  in  the  previous  passages,  I would  like  to  propose  a new  approach  of 
design — design  WITHIN  to  elaborate  on  the  possibilities  of  fostering  a design  culture.  By  taking  the  design-within 
approach,  the  user-designers  could  be  engaged  in  the  design  process  through  which  their  thinking  about  design  and 
systems  are  undergoing  inner  revolution.  Such  inner  revolution  will  be  dynamically  interacting  with  the  outer 
learning,  instructional  and  educational  transformation. 

A design  philosophy  arises  from  inner  revolution 

To  cultivate  our  inner  revolution,  we  should  hold  a truly  holistic,  systemic  view  of  their  inner  system  and 
relate  our  inner  systems  with  outer  systems,  and  furthermore,  resolving  the  boundary  existing  in  between.  Such 
systems  thinking  aims  not  at  depicting  individuals’  and  organizations’  recurrent  behavioral  structure,  but  at 
integrating  one’s  inner  systems  and  outer  learning  and  educational  systems.  Such  systems  thinking  is  to  be 
cultivated  through  nurturing  our  inner  systems  which  encompasses  awareness,  insights  and  visions.  To  nurture  the 
growth  of  such  inner  system,  we  could  take  efforts  through  the  following  3 tasks: 

1.  Arousing  our  AWARENESS  of 

• our  design  thinking  & action  patterns/structures 

• personal  and  collective  design  thinking  & action  pattems/structures 

• the  embedded  causal  links  in  our  design  thinking  & action, 

2.  Deepening  our  INSIGHTS  of 

• the  multiple  relationships  of  self,  others,  world  and  universe* 

• the  multiple  relationships  of  cause  and  effect 

• complexity  and  uncertainties  of  human  world 

3.  Cultivating  our  VISIONS  of 

• integrating  self  with  others  through  mutual  growth 

• transcending  personal  limitation,  & vicious  causal  links 

• tracing  the  pattern  and  strucure  of  problems 

When  engaged  in  building  a design-within  culture,  the  user-designers  will  not  first  define  their  design 
product  or  determine  their  design  strategies.  Instead,  they  will  look  inwardly  through  individual  and  collective 
reflection  to  nurture  their  systemic  visions,  insights,  will  and  awareness  of  design.  By  doing  so,  uncovering  one’s 
design  belief,  value  and  philosophy  would  become  the  priority  of  design  action.  The  design  goals  will  shift  from 
enhancing  effectiveness  and  efficient  to  deepening  design  participants’  visions,  insights,  will  and  awareness.  In 
other  words,  the  design-within  approach  identifies  problems  as  those  arising  from  one’s  inner  systems  which  are 
intertwined  with  and  inseparable  from  the  problems  we  point  outwardly  to.  The  design-within  approach  is 
concerned  with  how  our  design  might  be  influenced  by  our  inner  systems,  and  how  we  might  reconceptualize  our 
views  of  self  and  others,  designers  and  users.  When  we  view  self  and  others,  designers  and  users  as  oneness,  we 
could  approach  design-within  from  a truly  systemic  spirit.  Such  systemic  spirit  would  better  engage  us  in 
crystallizing  our  ingrained  design  beliefs,  values  and  philosophies.  Through  such  crystallization,  we  could  leap  out 
from  our  habitual  design  thinking  and  action  patterns. 

Interconnectedness  of  inner  and  outer  systems  dynamics 

Indeed,  design-within  is  humanistic-based  oriented  toward  one’s  inner  realization,  rather  than  product- 
based  or  goal-based.  Design-within  is  an  approach  which  extends  from  inner  systems  to  outer  learning  systems,  and 
arrows  from  the  outer  learning  systems  back  to  our  inner  systems.  It  focuses  on  the  ongoing  system  dynamics 
among  our  inner  systems,  outer  systems  and  inter-systems.  It  is  through  such  inner-outer-inter  systems  dynamics 
that  users  and  designers,  or  the  so-called  user-designers,  might  cultivate  truly  systemic  thinking,  and  integrate  with 
one  another  as  an  interconnected  design  community.  If  designers  and  users  could  collaborate  with  one  another 
beyond  the  level  of  information  and  knowledge  sharing,  and  engage  themselves  in  such  inner  revolution  and  systems 
dynamics,  transforming  our  design  mindsets  and  the  reality  might  be  possible.  While  engaged  in  the  design-within 
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process,  users  and  designers  probe  their  philosophy  and  values  of  design  in  reflective  dialogue,  and  gradually 
cultivate  their  design-within  thinking.  Through  such  thought  engagement,  an  interactive  process  from  outward  to 
inward,  from  inward  to  outward  will  be  integrated.  This  might  foster  our  design  thinking  and  action  with  truly 
systemic  spirit  so  as  to  take  the  responsibility  for  caring  the  whole,  rather  than  improving  parts  of  the  system. 

It  is  very  likely  that  user-designers  could  leap  out  from  the  existing  systems  if  their  inner  growth  is  taken 
into  serious  consideration.  Because  the  design-within  approach  aims  at  collective  and  dialogical  engagement  of  all 
participant’s  inner  revolution.  It  focuses  not  on  methods,  tools,  or  models  to  create  products,  but  on  methods,  tools 
and  models  to  engage  all  participants  in  the  systemic  thinking  process  and  in  the  inner  revolution  process.  Its 
ultimate  goal  is  to  transform  the  existing  inner  and  outer  systems.  I contend  that  when  our  design  thinking  is  no 
longer  locked  in  product-oriented  design,  we  might  foresee  the  magnificence  of  design  art.  The  design-within 
approach  is  to  explore  the  alternative  design  possibilities  for  transforming  both  inner  and  outer  world.  It  is  feedward 
from  and  feedback  to  the  inner  systems  which  are  integrated  with  any  outer  systems  we  attempt  to  design. 

The  roles  of  designers  in  the  design-within  approach 

Before  elaborating  on  what  roles  the  designers  in  the  “WITHIN”  approach  might  play,  it  is  helpful  to 
understand  how  Banathy  compare  the  roles  designers  might  play  in  different  modes  of  design.  The  following  table 
is  a brief  summary  of  Banathy’s  comparison  of  the  evolutionary  roles  of  designers. 


The  lsl  generation 
Designer 

* enters  a system  as  an  outside  expert 

* creates  an  image  of  the  future  system 
* hands  it  over  to  the  clients  for  implementation 

The  2nd 

generation  designer 

* has  slightly  more  interaction  with  clients 
* asks  clients  for  feedback  on  the  final  draft  before  turning  it  over 
to  them  for  implementation 

The  3rd 

generation  designer 

* enters  an  organization  as  outside  expert 
* draws  up  plans  for  the  design  or  redesign  of  a system 
with  even  more  interaction  with  clients. 

* invites  input  and  feedback  from  the  clients  throughout  the  design  process  and 
even  assist  them  with  the  implementation. 

The  4,h 

generation  designer  * fu 

takes  a radically  different  role  in  the  design  process 
ictions  primarily  as  a learning  facilitator  to  help  clients  learn  to  design  for  themseh 

* shifts  primary  responsibility  for  learing  and  designing  to  the 

clients 

* shoulders  responsibility  for  fostering  participants’  design 

competence 

* “does  with”  the  clients,  instead  of  “does  to”  or  “does  for”  the  clients 
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New  roles  of  designers— Visioning  designers 

Compared  to  the  four  generations  of  design  which  focus  on  “design  to”,  “design  by”,  “design  for”,  and 
“design  with”  respectively,  the  design-within  approach  places  emphasis  on  all  design  participants’  engagement  in 
design  “WITHIN”.  It  is  a thought-engagement  and  action-engagement  process  undertaken  either  individually  or 
colllectively.  Its  systemic  spirit  builds  on  the  integration  of  our  inner  systems  into  the  outer  systems,  and  the  inter- 
systems. It  is  such  holistic  system  dynamics  that  designers,  users  and  all  design  participants  are  engaged  in  the 
design  process  as  “visioning”  teams  or  design  communities,  rather  than  “visible”  teams  or  design  communities. 
Through  such  visioning  design  teams  or  community,  both  inner  revolution  and  societal  revolution  could  be  fostered. 
Designers  of  this  approach  share  some  similarities  with  the  4lh  generation  designers.  They  also  play  as  learning 
facilitators  to  help  clients  learn  to  design  for  themselves.  While  the  responsibility  they  need  to  take  is  fostering 
participants’  design  competence,  the  responsibility  for  learning  and  designing  is  shifted  to  the  clients.  Despite  these 
similarities,  they  would  play  other  significant  roles  which  the  4lh  generation  designers  may  even  be  unaware  of. 
Below  are  the  different  roles  they  would  play: 

• enters  the  design  systems  by  treating  themselves,  other  design  participants  and  the  social  systems  as  “systemic 
oneness” 

• cultivates  all  design  participants’  inner  growth  and  mutual  growth 

• engages  all  design  participants  in  reflective  and  critical  thinking  of  their  habitual  thinking  and  action  about 
design 

• functions  as  learning  enablers  by  transforming  design  into  an  inner  and  societal  learning  process 

• shifts  design  focus  from  design  product  to  design  process  to  “design  mind”,  a truly  humanistic-based  design 

In  one  word,  the  design-within  approach  focuses  on  cultivating  one’s  mind  for  design.  It  is  truly  systemic 
design  in  which  learning  and  design  comes  to  be  oneness,  everyone  is  working  toward  oneness  of  the  whole  system. 
Oneness  becomes  the  philosophical  foundation  for  constructing  an  integrated  design  team  or  community.  Through 
such  integration,  design  product  and  process  can  be  greatly  enriched  and  all  participants’  awareness,  vision,  insights 
and  will  of  transforming  reality  by  design  can  be  nurtured  and  cultivated. 

Conclusion 

The  knowledge  base  of  instructional  design  models  and  theories  encompasses  behaviorism,  neo- 
behaviorism, cognitivism,  constructivism,  situated  theory,  and  even  thinking  from  critical  theory  and 
postmodernism.  It  also  integrates  various  views  and  concepts  of  learning,  knowledge  and  systems  thinking.  Along 
with  the  evolution  of  design’s  knowledge  base,  the  roles  instructional  designers  play  also  change.  When 
instructional  designers  work  as  an  outside  consultant,  their  role  is  to  design  for  the  users,  working  at  best  as  the 
partners  of  subject  matter  experts,  or  the  design  experts  of  instructional  materials  or  activities.  When 
constructivism  and  situated  learning  gain  more  attention,  instructional  designers  are  expected  to  design  with  more 
comprehensive  and  timely  understanding  of  the  users’  needs  and  feedback.  They  are  also  expected  to  create  rich 
environments  and  situations  for  the  users’  learning,  reflection  and  action  (Streible,  1991).  When  embracing  the 
critical  paradigm  of  thinking,  instructional  designers’  responsibility  is  to  empower  the  learners’  critical  awareness  of 
their  own  thinking  and  action  so  that  learners  are  enabled  to  design  their  own  learning  (Li,  1993).  In  reviewing  the 
evolutionary  trend  of  instructional  design,  I found  that  different  orientations  of  instructional  design  imply  different 
design  philosophies  of  and  approaches  to  design.  Although  the  previous  research  efforts  have  consolidated  the 
knowledge  base  of  instructional  design  to  a certain  degree,  the  essence  of  instructional  design  is  yet  to  be  fully 
explored.  It  is  not  until  the  early  1990s  that  the  nature  of  design  thinking  and  systems  thinking  receives  more 
attention  in  the  field  of  instructional  technology;  and  more  scholars  share  with  the  vision  of  transforming  reality  by 
design.  As  instructional  designers,  we  need  to  leap  out  from  our  technical  mode  of  thinking  and  acting.  We  need 
systemic  change  in  the  way  we  think  and  act  about  learning  so  as  to  foster  our  inner  and  social  revolution.  Most 
important  of  all,  we  need  to  cultivate  strong  willingness  to  redefine  our  roles  as  teachers,  learners  or  designers.  We 
should  learn  to  be  more  responsible  for  our  own  learning,  and  resume  the  responsibility  to  design  our  own  learning 
in  the  forthcoming  learning  world. 
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Abstract 

This  research  examines  the  feasibility  of  rapid  application  development  (RAD)  in  the  design  of  two 
undergraduate  pre-service  teacher  technology  courses.  The  study  was  conducted  over  a three-semester  period 
during  which  a needs  assessment  was  conducted;  eleven  units  of  instruction  were  prototyped,  tested,  modified, 
retested;  and  the  course  was  fully  implemented.  The  total  number  of  participants  included  570  students,  six 
instructional  designers/teaching  assistants,  one  lead  designer,  and  four  lab  assistants.  Results  of  the  study  showed 
significant  differences  in  usability  scores  between  the  second  and  third  semester.  Both  positive  and  negative 
experiences  with  RAD  are  discussed. 

Introduction 

With  technology  changing  rapidly,  instruction  on  its  use  must  change  rapidly  as  well.  Instructional 
designers  are  thus  increasingly  dependent  on  the  use  of  rapid  application  development  (RAD)  procedures  for 
producing  timely  instruction  related  to  technology  use.  In  recent  years,  the  use  of  self-paced  instruction  in  the 
educational  environment  has  been  gaining  popularity  for  teaching  technology  skills.  A growing  number  of  self- 
paced  instructional  formats  are  now  readily  available  including  books,  tutorials,  and  video  series,  to  name  a few.  The 
benefits  of  self-paced  instruction  reach  an  increasingly  larger  audience  as  more  people  take  part  in  distance  learning 
environments,  which  may  in  whole  or  in  part  employ  self-paced  features.  Despite  the  benefits,  research  shows  that 
there  remains  a stigma  surrounding  teaching  a course  via  a ’distance'  format  (Betts,  1998).  Further  investigation 
shows  that  much  of  this  reticence  comes  from  the  initial,  up-front  time  investment  and  the  work  involved  by  the 
person(s)  designing  and  teaching  the  course  (Wollcott  & Betts,  1999,  p.  35-6). 

In  these  diverse,  fast-paced,  and  quickly  changing  educational  settings,  the  use  of  RAD  procedures  in 
creating  highly  usable  self-paced  instruction  is  advantageous.  How  then  do  we  implement  RAD  for  self-paced 
instruction,  and  how  do  we  make  the  self-paced  instruction  effective,  efficient,  and  appealing,  and  how  do  we  do 
that  in  short  order?  This  study  addresses  these  overarching  questions. 

Review  of  the  Literature 

In  the  1960s,  the  personalized  system  of  instruction  (PSI)  method  of  self-paced  instruction  was  developed 
by  Fred  Keller  [Keller  & Sherman,  (1982)].  PSI  was  designed  and  tested  to  address  researchers'  concerns  that 
teaching  was  primarily  focused  on  the  teachers  both  actively,  and  passively  presenting  information  to  a passive,  and 
at  times  somnolent,  student.  According  to  Sherman  (1992),  this  type  of  teaching  tends  to  neglect  what  and  how  the 
student  learns.  It  also  assumes  that  the  student,  as  a passive  learner,  will  one  way  or  another  receive  and  understand 
the  information  simply  because  the  information  has  been  presented.  Teaching  styles  during  that  time  also 
encouraged  negative  reinforcement  as  behavior/leaming  modification.  Sherman  goes  on  to  state  that,  “if  a high 
frequency  of  behavior  is  to  be  encouraged  so  that  progress  can  be  selectively  rewarded,  punishing  errors  is  the 
wrong  way  to  go  about  it”  (p.59).  Both  PSI  and  self-paced  instruction  address  these  concerns. 

According  to  Kemp,  Morrison,  & Ross  (1998),  PSI  is  also  often  referred  to  as  “the  Keller  Plan”  and  is  often 
used  when  planning  and  implementing  a whole  course  of  instruction.  At  the  time  of  its  inception,  this  method  was 
based  on  the  use  of  a textbook,  with  assigned  readings,  followed  by  questions  and  problem  solving.  With  the  advent 
of  microcomputers  and  especially  when  these  are  connected  to  the  Internet,  the  Keller  Plan  can  be  more  creatively 
applied — often  with  dramatically  positive  results. 

Self-paced  instruction  has  long  been  used  to  teach  computer  applications  such  as  word  processing, 
spreadsheets,  MS  Power  Point,  etc.  Technology,  however,  is  changing  so  rapidly  that  it  often  requires  frequent 
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updates  to  training  materials.  Unfortunately,  self-paced  materials  are  time  consuming  to  produce  and  thus  present  a 
developmental  challenge  in  a dynamic  technological  world.  Even  though  the  designing  of  self-paced  instruction 
brings  with  it  several  unique  challenges,  there  are  several  advantages  to  self-paced  instruction  that  make  it  worth  of 
the  time  expended  in  development  (Hannifin  & Peck,  1987).  Most  importantly,  self-paced  formats  allow  individuals 
to  progress  though  the  material  at  their  own  rate.  In  the  classroom  environment  frees  the  teacher  to  help  those 
students  who  need  augmented  and/or  more  ancillary  instruction.  In  both  the  classroom,  and  the  distance-learning 
environments,  self-paced  instruction  allows  the  learners  to  access  the  material  at  times  that  fit  their  individual 
schedules. 

It  is  common  practice  for  software  development  environments  to  follow  RAD  models  in  creating  their 
software  products  (Galitz,  1997).  Although  research  has  been  conducted  on  self-paced  instruction,  there  is  little 
specific  research  on  the  development  of  self-paced  instruction  using  RAD  procedures  in  academic  environments. 

Rapid  application  development  is  a term  used  to  describe  production  of  software  and  training  products  in  a 
series  of  development  cycles  that  move  quickly  from  initial  prototypes  to  a fully  developed  product.  In  these 
scenarios  a prototype  of  the  software  is  quickly  created,  tested  for  usability,  and  then  revised.  This  cycle  is  repeated 
until  the  final  product  has  been  developed.  Reigeluth  & Nelson  (1997)  describe  a RAD  approach  in  an  emerging 
paradigm  of  instructional  systems  design  (ISD)  they  call  ASEC  (Analysis,  Synthesis,  Evaluation,  and  Change). 
Design  and  development  phases  move  through  an  ’’iterative  series  of  ASEC  cycles  for  progressive  sets  of 
instructional  decisions”  (Reigeluth  & Nelson,  1997,  p.  31). 

The  ASEC  process  allows  a designer/developer  to  focus  on  how  a learner  or  user  responds  to  instruction  as 
it  is  being  developed.  The  Evaluation  Step  in  the  ASEC  process  allows  for  this  type  of  leamer/user  feedback, 
(important  to  learning  more  about  the  usability  or  learner-friendliness  of  the  product).  "User-friendly"  is  a term  that 
describes  how  technically  easy  a product  is.  "Learner-friendly"  describes  how  easy  an  instructional  product  is  for  a 
learner  to  use  (Lohr,  2000).  The  International  Standards  Organization  (ISO)  defines  usability  with  the  descriptors 
effectiveness,  efficiency,  and  satisfaction.  Lohr  (2000)  defines  usability  and  learner-friendliness  with  effectiveness, 
efficiency,  and  appeal,  based  on  similar  definitions  by  other  design  researchers  ( Flagg,  1990;  Nielson,  1993; 
Shackel,  1991;  Tessmer,  1993) . 

Research  Questions 

This  study  focuses  on  the  development  of  self-paced  pre-service  teacher  technology  training  using  a RAD 
approach.  Research  questions  include: 

1 . How  usable  was  the  instruction  from  student,  instructional  designer/Teaching  Assistant  (ID/TA),  lead  designer, 
and  lab  assistant  perspectives? 

2.  Did  usability  perceptions  change  during  the  RAD  process? 

3.  What  are  the  advantages  and  disadvantages  of  using  a RAD  process  in  an  academic  setting? 

Methodology 

Participants 

This  research  was  conducted  at  a medium-sized  Western  university  and  involved  570  pre-service  teachers 
enrolled  in  two  levels  of  technology-integration  courses.  Other  participants  in  the  study  included  six  instructional 
designers/teaching  assistants  (IDs/TAs),  a lead  instructional  designer,  and  four  lab  assistants.  Out  of  657  total 
students  enrolled  in  both  200-level,  and  300-level  levels  during  the  two  semesters,  a total  of  570  students 
participated  in  the  study.  Of  the  570  students  who  filled  out  the  surveys,  316  students  were  from  the  200-level 
classes  and  254  students  from  300-level  classes  (described  later  in  this  paper).  At  the  end  of  each  semester,  four 
students  from  each  class  were  randomly  selected  to  be  interviewed  (a  total  of  118). 

Course  Description 

Two  one-hour  pre-service  teacher  technology  courses  were  developed  for  this  study,  referred  to  as  the  200- 
level  and  300-level  courses.  The  200-level  course  was  an  introduction  to  technology  for  freshmen  or  sophomores. 

The  300-level  course  was  a more  advanced  course  for  juniors  and  seniors.  In  both  courses  combined,  there  were  a 
total  of  1 1 units  of  instructions  to  be  created.  The  topics  included,  email,  uses  of  the  World  Wide  Web  (WWW),  and 
word  processing. 

All  sections  were  taught  in  three  formats:  workshops,  open-labs,  and  via  self-paced  instruction.  The  three 
formats  broke  down  as  follows:  The  workshops  consisted  of  instruction,  both  verbal  and  visual,  where  each  project 
was  completed  in  front  of  the  students.  In  this  format  many  of  the  students  could  follow  along  step  by  step.  The  open 
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labs  were  set  up  so  that  any  student  could  get  individual  instruction.  For  the  aspects  of  each  project  where  a number 
of  students  were  struggling  over  the  same  component,  this  was  noted  so  that  it  could  be  addressed  in  a redesign 
between  semesters.  The  lab  assistants  were  also  present  during  open  labs  to  help  students  one-on-one  with  their 
individual  difficulties.  The  self-paced  instruction  was  taught  via  the  web  sites  set  up  for  each  of  the  two  courses. 
Every  student  was  required  to  have  an  e-mail  account  and  access  to  the  Internet.  The  students  could  access  the  200- 
level  and  300-level  web  sites  where  all  the  1 1 units  of  instruction  were  hosted.  To  view  these  units  of  instruction, 
visit  these  Web  sites;  www.edtech.unco.edu/et34x/  and  www.edtech.unco.edu/et24x/. 

Procedures 

The  study  was  conducted  over  three  semesters  (Spring  of  1999,  Fall  of  1999  and  Fpring  of  2000).  The 
topic  covered  as  well  as  the  formats  of  instruction  were  gleaned  from  a pre-service  teacher  needs  assessment 
conducted  in  the  spring  semester  of  1999.  Three  students  (pre-service  teachers),  three  graduate-level  teaching 
assistants,  and  one  faculty  member  were  interviewed  about  the  effectiveness,  efficiency,  and  appeal  of  the 
undergraduate  pre-service  teacher  technology  courses  at  that  time.  These  interviews  revealed  the  following:  a)  there 
existed  a lack  of  instructional  consistency  in  courses  (different  instructors  teaching  different  topic  in  different  ways); 
b)  the  instructional  materials  were  outdated;  c)  students  came  to  the  class  with  different  proficiency  levels  ranging 
from  computer  illiterate  to  highly  skilled  computer  users,  but  all  students  were  expected  to  learn  at  the  same  pace;  d) 
students  were  required  to  attend  all  classes  even  if  they  knew,  and  were  proficient  with,  the  software  being  taught;  e) 
many  students  had  scheduling  conflicts  that  could  have  been  prevented  with  the  implementation  of  a self-paced 
format. 

The  needs  assessment  team  identified  self-paced  instruction  as  a possible  solution  to  these  issues,  since  SPI 
not  only  allows  for  learners  to  progress  at  their  own  pace,  but  allows  for  greater  standardization  of  the  instructional 
content  as  well. 

Beginning  with  the  fall  semester  of  1999,  a team  of  three  teaching  assistants  ( TA’s)  and  a lead 
instructional  designer  (ID)  planned  the  development  and  implementation  of  RAD  to  the  instructional  material 
needed  for  the  courses.  The  team  met  weekly  intervals  to  plan  and  review  their  work,  discuss  problems,  concerns, 
and  progress.  The  team  also  kept  a record  of  their  observations.  The  actual  RAD  course  development  was 
performed  throughout  the  first  semester,  but  a revising  process  (implementing  feedback  from  the  first  semesters’ 
participants)  continued  between  the  semesters  as  well.  Each  of  the  teaching  assistants  developed  approximately 
one-third  of  the  total  units  of  instruction. 

Every  week  each  of  the  developed  units  of  instruction  would  be  presented  to  the  team  members  who  would 
collectively  review  what  each  TA  had  designed,  test  the  instruction,  and  make  suggestions.  At  this  time  the  lead  ID 
would  also  make  suggestions.  After  this  open-forum-type  of  critiquing  was  finished,  the  TA  in  charge  of  each  unit 
of  instruction  would  make  the  necessary  revisions,  test  it,  and  then  re-present  it  to  the  group  the  following  week 
where  the  rest  of  the  team  members  would  again  review  the  TA's  work,  usability  test  the  unit  of  instruction,  and  give 
their  final  suggestions.  The  TA  would  then  make  any  necessary  final  revisions,  transfer  the  finalized  unit  of 
instruction  to  the  course  web  site  for  those  who  were  doing  the  course  via  self-paced  instruction,  and  the  instructor 
would  also  present  it  to  the  class  as  an  in-class  instruction  module.  This  process  of  designing,  developing,  open- 
forum  critiquing,  revising,  and  usability  testing  was  followed  for  each  of  the  1 1 units  of  instructions.  Thus,  every 
two  weeks  and  within  a two-week  span,  the  RAD  model  was  used  with  two  individual  units  of  instruction. 

Instruments 

Quantitative  and  qualitative  data  were  collected  from  the  following  sources:  students  participating  in  the 
course;  teaching  assistants,  all  of  whom  not  only  taught  the  course,  but  also  monitored  the  lab  sessions,  and  some  of 
whom  (instructional  designers)  also  helped  design  the  course;  the  lead  instructional  designer  who  managed  the 
design  and  development  process,  and  the  lab  assistants  who  interacted  with  the  students  while  the  students 
participated  in  the  instruction.  Both  the  lead  instructional  designer  and  the  IDs/TAs  kept  journals  for  a 12-week 
period.  Lab  assitants  were  informally  interviewed.  Qualitative  student  data  was  obtained  using  16  open-ended 
questions  included  in  an  end-of-the-semester  questionnaire,  from  120  student  observations,  and  from  118  student 
interviews. 

All  qualitative  data  were  transcribed,  and  thoroughly  read  and  re-read.  Different  colored  markers  were  used 
to  highlight  words  and  statements  which  fell  into  the  three  main  research  categories:  effectiveness,  efficiency,  and 
appeal.  An  overall  theme  and  sub-  were  then  identified  for  each  category. 

Quantitative  data  were  collected  using  43  Likert-type  items  included  in  an  end-of-the-semester 
questionnaire.  The  five-point  Likert  scale  ranged  from  “strongly  disagree”  to  “strongly  agree.”  The  items  were 
created  for  each  of  the  three  main  domain  areas:  effectiveness,  efficiency,  and  appeal.  Factor  analysis  was  used  to 
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verify  the  structure  of  the  above  mentioned  survey  questionnaire.  A structure  matrix  of  a three-factor  oblique 
rotation  (Kappa  = 4)  of  a principle-components  solution  was  found.  The  factors  were  learning  and  educational  value 
(effectiveness),  resources  (efficiency),  and  self-paced  instruction  (appeal).  Problematic  items  i.e.,  loaded  on  two 
factors  or  demonstrated  a factor  loading  under  .30  on  any  factor,  were  not  used  as  part  of  the  composite  scores.  Item 
analysis  within  each  factor  demonstrated  strong  reliability,  and  hence  stability  of  the  survey  as  a whole.  Reliability 
coefficients  for  the  200-level  courses  were  found  to  be  corrective  = 0.9301,  oornciency  = 0.8071  and  appeal  = 0.8070 
respectively.  Reliability  coefficients  for  the  300-level  courses  were  found  to  be  00frective=  0.9121,  00fnciency = 0.7270 
and  otappeai  = 0.8775  respectively.  The  final  composition  of  the  domains  suggest  a degree  of  validity. 


Results 

Students  perceptions  of  usability 
Effectiveness 

Qualitative  analysis  of  the  randomly  selected  interviews  and  open-ended  questions  found  a strong  majority 
of  positive  responses  from  students  related  to  the  effectiveness  of  the  course.  Students  mentioned  liking  the  course 
for  a variety  of  reasons  that  were  categorized  as  educational  and  learning  value.  Data  were  coded  into  the  following 
sub-themes:  content  learned  was  applicable  to  the  classroom;  knowledge  was  gained;  portfolios  were  developed; 
topics  could  be  freely  chosen;  projects  were  relevant;  the  overall  format  was  convenient;  he  experience  was  isolating 
and  often  not  challenging  enough.  Most  of  the  positive  comments  from  both  levels  were  associated  with  how  precise 
the  instruction  was  and  how  useful  it  would  be  in  a classroom.  Most  of  the  negative  comments  were  associated  with 
dislike  for  specific  topics  that  were  considered  repetitive  (ASSURE),  too  difficult  (Photoshop),  or  too  easy  (email). 
The  most  common  statements  used  by  students  were  as  follows:  “I  learned  a lot  in  this  class.”;  “I  gained  more 
knowledge,  new  ideas,  and  ways  to  use  technology.”;  and,  “I  feel  comfortable  with  technology.”  All  of  the  students 
randomly  selected  to  perform  a task  representing  the  semester’s  work  performed  satisfactorily.  All  of  these  same 
students  felt  the  textbooks  were  a "waste  of  time". 

Quantitative  data  for  effectiveness  was  calculated  by  averaging  the  percentage  of  agree  and  strongly  agree 
scores  for  fall  and  spring  semester  for  each  course  level  on  specific  questions.  Quantitative  results  were  similar  to 
qualitative  results  with  a strong  majority  of  students  (86%  of  fall  and  spring  200-level  and  72%  of  the  fall  and  spring 
300-level)  agreeing  or  strongly  agreeing  that  the  course  was  educationally  valuable.  Eighty  nine  percent  of  the  200- 
level  and  72%  of  the  300-  level  agreed  or  strongly  agreed  that  skills  learned  were  important.  Fewer  students 
indicated  they  learned  a lot  (61%  of  the  200-level,  69%  of  the  300-level).  As  indicated  by  the  qualitative  data  as 
well,  both  levels  felt  that  the  instruction,  given  both  in-class  and  on-line,  was  precise  and  clear.  These  students 
indicated  that  they  would  use  the  technological  skills  learned  during  this  course  both  in  their  future  classrooms  for 
instructional  delivery,  and  for  their  own  personal  classroom  management  use  and  gains.  They  stated  that  this  usage 
would  include  programs  like  PowerPoint  (200-level)  and  Web  design  (300-level,  also  suggested  as  a way  to  gain 
employment)  and  incorporate  the  design  and  graphics  knowledge  they  had  learned.  These  students  indicated  that  the 
course  helped  them  not  only  learn  skills  with  which  they  were  previously  unfamiliar,  but  also  to  learn  how  to 
integrate  technology  into  their  classrooms.  At  the  end  of  the  course,  a majority  of  students  felt  very  comfortable  with 
technology 

Efficiency 

In  the  Qualitative  analysis  the  Value  of  resources  in  terms  of  access  and  ease  of  use  emerged  as  the  key 
theme  or  descriptor  for  questions  related  to  efficiency.  Data  were  coded,  then  grouped  into  these  sub-themes:  online 
materials,  open  labs,  workshops,  instructors  and  lab-consultants.  Two-hundred-level  and  300-level  students  were 
most  positive  about  having  a choice  of  learning  format  (workshop,  open-lab,  or  self-paced),  and  most  negative  about 
specific  projects  and  isolation.  The  300-level  group  felt  strongly  that  they  had  too  much  work  to  do  for  a one-credit 
course.  Of  those  who  were  interviewed  that  did  not  like  the  course,  two  main  reasons  were  given.  Some  mentioned 
they  did  not  like  the  self-paced  format  because  of  isolation  and  problems  with  self-discipline.  Others  mentioned  they 
didn't  believe  the  technology  would  be  available  to  them  in  the  classroom.  All  students  felt  the  textbooks  were  not 
helpful.  Half  of  the  students  in  the  first  semester  wanted  more  project  examples  when  interviewed  at  the  end  of  the 
fall  semester.  Surprisingly  just  a little  under  a half  of  the  students  interviewed  in  the  spring  still  indicated  the  need 
for  more  examples  (despite  the  development  of  examples  and  non-examples  for  each  unit  of  instruction  that  took 
place  during  the  spring  semester). 
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Typical  comments  during  student  interviews  included:  “The  on-line  materials  were  precise  and  easy  to 
understand  because  of  the  step  by  step  instructions”;  “The  rubrics/or  grading  criteria  were  too  easy”;  “I  would  like  to 
see  more  examples  of  finished  projects”  ;“PhotoShop  was  too  confusing”;  “In  open-labs  I could  get  personal 
attention  and  help  to  complete  my  projects”;*4  Workshops  were  very  helpful  for  difficult  projects  like  PhotoShop, 
Web  design  and  PowerPoint”;  “The  instructors  were  very  approachable,  accessible  and  always  available  to  help 
me”;  “The  instructors  were  knowledgeable. ”;“The  lab  consultants  were  there  to  help  me  when  I got  stuck.” 

Quantitative  data  for  efficiency  was  calculated  by  averaging  the  percentage  of  agree  and  strongly  agree 
scores  for  fall  and  spring  semester  for  each  course  level  on  specific  questions.  Quantitative  data  was  similar  to 
qualitative  with  a higher  percentage  of  neutral  responses  and  a lower  percentage  of  responses  in  the  agree  and 
strongly  agree  categories.  Approximately  60%  of  the  200-level  responses  were  positive  about  both  the  instructors 
and  the  self-paced  instruction.  Approximately  75%  of  the  300-level  responses  were  positive  about  the  instructors 
and  the  self-paced  instruction.  Open  labs  and  workshops  received  lower  scores,  with  less  than  half  of  the  200-level 
students  positive  about  either  format.  The  data  was  similar  for  the  300-level  who  had  a slightly  higher  percentage  of 
agree  or  strongly  agree  responses  (59%  for  workshops  and  55%  for  open-labs.)  Of  particular  interest  is  that  only 
58%  of  the  200-level  and  59%  of  the  300-level  felt  they  had  enough  self-discipline  for  the  self-paced  format. 

Appeal 

Of  the  three  usability  variables,  appeal  had  the  most  positive  student  responses.  Quantitative  data  related  to 
appeal  was  more  positive  than  the  qualitative  data  regarding  overall  course  appeal.  From  the  qualitative  data,  a 
a strong  majority  of  students  interviewed  mentioned  liking  the  self-paced  nature  of  the  course.  Not  surprisingly,  the 
key  theme  that  emerged  from  the  questions  was  a positive  regard  for  the  self-paced  format.  Self-paced  thus  emerged 
as  a key  descriptor  for  appeal  based  on  data  that  was  coded  then  grouped  into  following  sub-themes:  freedom,  self- 
discipline,  convenience,  isolation,  and  not  challenging.  Both  levels  of  students  mentioned  the  self-paced  format  gave 
them  the  freedom  and  flexibility  to  go  at  their  own  rate  and,  hence,  made  them  feel  less  pressured  and  felt  they  had  a 
lower  stress  level  than  if  they  had  taken  the  course  via  a traditional  in-class  format.  The  self-paced  format  was 
particularly  helpful  for  the  students  who  already  knew  most  of  the  tools.  The  self-paced  format  also  required 
students  to  apply  better  time-management  skills.  Again,  some  students  did  not  like  the  self-paced  because  they  felt 
the  format  lacked  student/teacher  interaction  as  well  as  class-related  communication  with  fellow  classmates.  Typical 
comments  included:  “I  liked  the  option  to  attend  workshops  open  labs  or  follow  self-paced  for  projects  that  I was 
comfortable  with”;  “It  was  nice  to  know  that  help  was  available  whenever  required”;  “I  learned  to  manage  time 
effectively”;  “I  sometimes  tended  to  procrastinate  my  work”;  “I  liked  the  convenience  of  working  at  my  own  time 
and  pace”;  “I  liked  the  fact  that  I did  not  have  to  come  to  the  class  if  I knew  the  project”;  “The  fact  that  I did  not 
meet  officially  made  me  feel  isolated”;  “Projects  like  word,  e-mail,  WWW  were  too  simple  and  basic”;  “I  knew 
most  of  the  software  and  hence  was  hard  to  stay  motivated.” 

Quantitative  data  for  appeal  was  calculated  by  averaging  the  percentage  of  agree  and  strongly  agree  scores 
for  fall  and  spring  semester  for  each  course  level  for  specific  questions.  Results  related  to  appeal  were  quite  high  for 
both  the  200  and  300  level  students.  Approximately  90  % of  the  students  agreed  or  strongly  agreed  to  statements 
regarding  the  overall  usefulness  of  self-paced  instruction.  For  example  97%  of  200-level  students  and  89%  of  300 
level  students  indicated  liking  self-paced  instruction.  Ninety  five  percent  of  the  200-level  students  and  89%  of  the 
300-level  students  indicated  the  self-paced  instruction  was  helpful. 

Teaching  Assistants/Instructional  Designers 

The  instructional  designer/teaching  assistants  were  asked  to  keep  a journal  during  the  fall  semester  to 
record  two  types  of  information:  1)  student  behaviors  and  attitudes,  and  2)  issues  relating  to  the  format  and  accuracy 
of  instruction.  Perceived  effectiveness,  efficiency,  and  appeal  data  were  categorized  and  grouped.  Observations 
related  to  the  effectiveness  of  the  course  included  noticing  that  students  were  often  copying  the  assignment  example 
instead  of  generating  their  own  solutions,  and  were  not  checking  the  Web  site  bulletin  board.  Observations  related  to 
the  efficiency  included  the  lack  of  student  attendance  at  workshops.  For  example,  students  would  elect  the  self- 
paced  format,  wait  until  the  last  minute,  and  then  discover  they  could  not  do  the  assignment  without  help. 
Observations  related  to  appeal  included  reports  that  many  students  were  unhappy  with  scheduling  issues.  These 
students  said  the  classes  did  not  meet  during  convenient  times. 

Several  actions  were  taken  immediately  prior  to  the  next  semester  to  address  these  issues;  1)  the  free-for-all 
schedule  was  eliminated  requiring  students  to  sign  up  for  a class  at  a specified  time;  2)  students  were  required  to 
attend  workshops  unless  they  had  turned  in  the  self-paced  instruction;  and  3)  class  listserves  were  used  as  a 
communication  vehicle  in  addition  to  the  message  board.  During  the  following  semester  these  problems  were  not 
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mentioned  by  the  students.  One  issue  however  that  remained  a problem  was  the  need  for  the  Web  site  to  provide 
more  and  varied  project  examples  since  students  continued  to  turn  in  assignments  very  similar  to  the  examples  used. 

Lab  Assistants 

The  lab  assistant  interviews  indicated  mostly  problems  with  efficiency  and  effectiveness.  Lab  assistants 
described  themselves  as  being  bombarded  the  day  an  assignment  was  due.  These  students  felt  that  they  had  to  teach 
students  how  to  do  their  projects.  The  interviews  revealed  that  over  half  of  the  lab  assistants  did  not  know  about  the 
self-paced  materials  or  where  they  could  be  located.  Those  who  did  know  indicated  that  most  of  the  students  did  not 
have  the  web-site  address  easily  accessible.  Lab  assistants  addressed  this  problem  by  writing  the  class  website  on  lab 
whiteboard.  The  one  problem  identified  in  the  Spring  semester  interviews  with  lab  assistants  was  the  difficulty 
students  were  still  experiencing  with  the  Web  site  URL.  Therefore  in  the  third  semester  of  the  project,  the  web  site 
was  given  a new  and  simpler  URL,  and  desktop  Web  site  icons  were  created  to  allow  students  to  access  the  site  by 
clicking  the  icon  instead  of  typing  in  a lengthy  url  (http://www.edtech.unco.edu/et34x/def34x.html). 

The  Lead  Instructional  Designer 

The  Analysis,  Synthesis,  Evaluation,  and  Change  interpretation  of  RAD  was  documented  and  evaluated  on 
a weekly  basis  by  the  lead  designer  to  determine  overall  effectiveness  of  the  RAD  approach.  Time  spent, 
management  issues,  technical  issues,  and  student  behaviors  were  noted  for  all  eleven  cycles  of  development  during 
the  fall  semester.  Of  each  of  the  steps,  the  lead  designer  was  most  disappointed  in  the  inability  to  implement  learner 
testing  in  the  Evaluation  step.  Lack  of  time  prevented  the  IDs  to  involve  students  in  the  development  of  the 
instruction.  Therefore  the  learner-testing  phase  was  actually  a design-team  testing  phase.  The  process  however 
seemed  adequately  effective  even  without  student  involvement  in  the  two-week  testing  phase.  The  units  of 
instruction  were  delivered  on  time  with  few  errors. 

The  efficiency  of  the  RAD  process  was  assessed  in  part  by  recording  time-spent  on  specific  activites.  The 
lead  designer  was  not  prepared  for  the  amount  of  time  it  took  to  set  up  the  course  templates  and  create  master 
schedules  for  open  labs  and  workshops  to  accommodate  student  schedules.  Three  full  days  were  spent  working  on 
the  schedule  alone.  These  cycles  included  a brief  analysis  step  (identification  of  a specific  task  and  content  needed), 
development  of  instruction  (seven  hours),  testing  by  the  team  (1-2  hours),  revision  and  Web  posting  by  the 
designer  (8  hours),  team  re-testing  and  revision,  changes  (3  hours),  and  implementation.  The  lead  designer  also 
regretted  her  decision  to  use  Microsoft  Word  as  the  Web-site  authoring  tool  because  it  was  less  stable  on  the  server 
than  were  other  Web  authoring  programs.  Microsoft  Word  was  chosen  because  most  IDs/TAs  were  not  proficient 
with  Web  authoring  and  needed  a very  simple  tool.  Unfortunately  this  backfired  with  numerous  server  problems  that 
took  up  an  additional  four  hours  per  week  to  address. 

The  more  interesting  aspect  of  the  appeal  data  relate  to  the  RAD  process  came  from  the  lead  designer's 
observations  of  ID's/TA's  gradual  acceptance  of  both  the  format  of  instruction  and  the  RAD  process.  Early  in  the 
first  semester  students  were  highly  frustrated  with  the  process  of  rapid  development.  They  were  unfamiliar  with  the 
precision  self-paced  instruction  required  as  well  as  the  demand  for  user-friendly  writing.  Combined  with  low 
workshop  and  open  lab  attendance  during  the  first  few  weeks  of  the  semester,  these  T A/IDs  were  openly  frustrated. 
By  the  middle  of  the  semester  they  voiced  extreme  dissatisfaction  with  the  process  and  structure.  During  the  second 
half  of  the  semester  they  gradually  gained  more  acceptance  of  the  format  and  showed  interest  in  making 
improvements.  Also  noted  was  their  improved  ability  to  write  learner-friendly  instruction. 

Change  In  Usability  Between  Semesters 

Lead  designer,  TAs/IDs,  and  lab  assistants  all  noticed  improvement  in  the  course  during  the  Spring 
semester.  These  observations  are  in  part  supported  by  multivariate  analysis  of  variance  (MANOVA)  results.  A 
MANOVA  was  used  to  investigate  differences  between  the  fall  1999  and  spring  2000  semesters  with  respect  to  the 
factors:  effectiveness,  efficiency  and  appeal.  Significant  MANOVA  results  were  followed  by  discriminate  analysis. 
A significant  difference  was  found  in  the  ET  200-level  data  between  the  fall  of  1999  and  the  spring  of  2000 
semesters  (F  (3,301)  = 8.356,  p<0.00,  Wilks’  lambda=0.923).  A stepwise  discriminate  analysis  was  used  to 
determine  if  there  was  a specific  variable  contributing  the  strongest  influence  to  this  difference.  It  was  found  that 
only  the  resources  factor  - efficiency  - (e.g.,  workshops,  open  labs,  and  self-paced  instruction)  that  contributed 
most  to  a change  between  semesters  (F  (1 ,303)  = 22. 1 1 , p<0.00). 

In  comparing  the  fall  and  spring  ET  300-level  courses,  significant  differences  in  all  three  factors  (F  (3,240) 
= 19.702,  p < 0.00,  Wilks’  lambda  = 0.802)  were  found.  A stepwise  discriminate  analysis  was  used  to  determine  if 
there  was  a specific  factor  contributing  the  strongest  influence  to  this  difference.  This  analysis  indicated  that  all  the 
three  dependent  variables  were  equally  responsible  for  the  differences  between  fall  of  1999  and  the  spring  of  2000 


225 


232 


semesters.  The  stepwise  discriminate  analysis  showed  that  Appeal  made  the  biggest  difference  among  semesters 
(F=45.568,  Wilks’  Lambda=.955).  Appeal  was  followed  by  Effectiveness  (F=41.42 6,  Wilks’  Lamda  = .941). 
Efficiency  made  the  smallest  difference  among  semesters  (F  = 8. 1 8,  Wilks’  Samda  = .83 1 ). 


That  significant  differences  were  found  in  usability  perceptions  between  the  second  and  third  semesters 
might  suggest  that  RAD  played  a role  in  improving  instruction.  Changes  to  instruction  were  made  based  on  data 
gathered  during  the  semester.  For  example  clarifications  in  assignments,  additional  examples,  the  correction  of 
errors,  all  improved  the  quality  of  the  experience.  Many  of  the  concerns  and  complaints  expressed  by  TAs  and 
students  alike  during  the  first  semester  were  not  voiced  during  the  second  semester. 

There  are  however,  many  possible  reasons  for  the  significant  differences  that  took  place  between  the 
second  and  third  semester  that  may  not  be  attributable  to  RAD.  A more  positive  or  motivated  group  of  students  may 
have  been  enrolled  in  the  second  semester.  TAs  may  have  conveyed  more  confidence,  thereby  influencing  the 
attitudes  of  the  students  for  a variety  of  reasons.  Most  obvious  would  be  that  the  materials  were  already  developed 
and  error- free,  therefore  TAs  had  more  time  to  devote  to  individual  students,  and  were  also  more  confident  that 
fewer  mistakes  would  cause  student  confusion.  The  simple  change  in  required  scheduling  had  perhaps  the  biggest 
impact  as  many  students  were  dissatisfied  with  the  open  schedule  format  of  the  first  semester.  This  schedule  change 
was  not  a function  of  RAD. 

The  key  benefits  of  RAD  in  this  study  appears  to  be  improved  instruction  that  lasts  for  a longer  period  of 
time.  There  has  been  much  cited  in  the  research  about  the  low  return  on  investment  of  self-paced  instruction 
(Wolcott  & Betts,  1999).  Initially,  in  this  study,  that  was  the  case  as  well;  however,  because  the  course  content  has 
remained  fairly  stable  over  the  past  year  and  a half,  the  time  expenditure  has  been  amortized  very  effectively.  As  this 
article  is  being  written  (halfway  through  the  third  semester  after  the  RAD  took  place)  the  researchers  have  found 
that  the  time  expended  for  evaluation  and  revision  have  been  substantially  less  than  if  the  courses  were  being  taught 
via  a more  traditional  format.  For  the  spring  2000  semester,  approximately  a week  was  spend  in  completing  a 
cursory  analysis  of  the  data  from  the  previous  semester’s  surveys;  in  reevaluating  the  course  philosophies  and 
content  in  light  of  the  surveys;  and  in  revising  the  Websites,  syllabi,  rubrics,  and  schedules.  In  the  summer 
semester,  there  were  no  changes  made  other  than  changing  the  dates  and  times  on  both  the  syllabi  and  the  course 
schedule,  which  took  approximately  two  hours  for  both  courses.  In  the  fall  2000  semester,  a total  of  approximately 
two  days  was  spent  in  the  review,  analysis,  and  revision  process.  All  told,  for  two  different  courses  that  serve  so 
many  students,  this  time  expenditure  is  quite  modest. 

The  key  drawback  of  RAD  in  this  study  was  the  need  to  act  quickly.  In  this  study  hasty  decisions  had  long- 
lasting  ramifications.  When  the  study  began,  the  lead  ID  drew  up  the  original  course  outline  and  even  created  the 
initial  Website  shell  for  the  courses  using  a very  simple  authoring  tool,  Microsoft  Word.  This  was  done  because  the 
TA/IDs  had  little  experience  with  developing  Web  based  instruction  and  would  need  a familiar  and  easy 
development  tool.  Because  the  Lead  ID  did  not  want  application  specific  ramp-up  time  to  impede  the  IDs  progress, 
it  was  decided  that  templates  could  be  created  and  used.  Because  Microsoft  Word  is  a fairly  basic  web-authoring 
tool  the  software  became  a limiting  factor  once  the  demands  of  the  site  require  more  complexity.  The  team 
recommends  beginning  with  a more  robust  web-authoring  software  capable  of  accommodating  a broader  range  of 
Web  site  maintenance  tasks,  such  as  frequent  revisions  to  instruction  and  use  of  different  browsers  and  Web  editors. 

Also  of  interest  to  future  RAD  projects  in  academic  environments  is  the  effect  the  approach  has  on  novice 
instructional  designers.  Because  rapid  application  development  is  just  that,  rapid,  it  brings  with  it  a set  of  dynamics 
with  which  most  academic  settings  are  not  usually  confronted.  In  this  study,  the  lead  ID  had  been  through  this  type 
of  development  numerous  times  and,  as  a result,  was  mentally  prepared  for  the  frustration  that  is  typically  a part  of 
the  designing,  developing,  open-forum  critiquing,  revising,  and  usability  testing  process.  Individual  and  team 
frustration  is  so  inherent  in  the  RAD  model  that  it  can  almost  be  mapped  out  and  defined  as  part  of  the  cycle.  In  this 
study’s  cycle,  the  TA/IDs  became  quite  frustrated  with  the  intensity  and  seeming  lack  of  progress  early  on.  However, 
as  the  process  continued  and  the  fresh,  never-tried-before  units  of  instruction  were  introduced  to  the  students  they 
seamlessly  learned  the  material  and  skills  and  produced  the  hoped-for  and  required  results.  The  TA/IDs  then  began 
to  more  clearly  understand  how  RAD  works,  and  as  a result,  became  much  more  enthusiastic  about  its  application 
and  results. 


Overall,  quantitative  and  qualitative  data  collected  from  students,  TAs/IDs,  the  lead  ID,  and  lab  assistants 
suggest  that  the  course  was  learner-friendly.  The  data  identified  areas  for  improvement  as  well  as  areas  where 
students  were  highly  satisfied  with  the  quality  of  instruction  in  both  the  200-level  and  300-level  courses.  The 
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importance  of  providing  instruction  that  is  neither  too  easy  or  too  difficult  was  identified,  as  both  were  mentioned  as 
negative  aspects  of  the  course.  The  need  to  help  students  think  creatively  and  find  ways  to  use  the  tool  for  their 
unique  instructional  purposes  must  be  improved,  perhaps  through  many  and  varied  examples  and  non-examples. 
Scheduling  problems  and  difficult  Web  site  access  negatively  influenced  student's  perceptions  of  efficiency  and 
appeal.  The  self-paced  format,  however,  was  considered  highly  appealing.  Though  many  students  considered 
themselves  self-disciplined,  the  data  suggested  the  need  to  implement  a structure  for  students  that  imposed  due 
dates,  designated  class  times,  and  the  requirement  to  have  self-paced  instruction  completed  prior  to  a workshop. 

Although  RAD  has  found  an  almost  hallowed  place  in  fast-paced  corporate  settings,  we  were  reticent  about 
its  usefulness  in  an  academic  setting.  The  results  of  this  study,  however,  have  shown  that  RAD  can  be  applied  quite 
effectively.  The  results  have  shown  that  in  the  right  application,  time-dependant  course  material  can  be  delivered 
efficiently,  effectively,  and  in  an  appealing  manner  using  RAD.  The  caveat  is  that  all  aspects  of  the  process  should 
be  well  thought  out,  critiqued,  revised,  and  usability  tested. 

Several  areas  of  future  research  related  to  this  topic  should  be  explored.  For  one,  the  importance  of 
instructional  graphics  needs  to  be  examined.  Because  instructional  graphics  are  so  time-consuming  to  develop,  an 
important  consideration  is  the  optimal  amount  of  graphics  to  include  as  well  as  the  type  of  graphics  for  RAD  of  self- 
paced  instruction.  For  example,  should  graphics  show  an  entire  computer  screen,  or  should  they  display  just  a button 
or  pull  down  menu?  Additionally,  what  are  the  most  learner-friendly  writing  conventions?  Is  "select  Save  As  from 
the  File  pull  down  menu"  as  effective  as  File  > Save  As?  Another  more  challenging  question  asks  how  pre-service 
teachers  can  be  taught  to  think  creatively  when  applying  technology  to  their  unique  instructional  management  or 
development  goals.  The  excuse  of  not  needing  to  learn  about  technology  because  one  already  knows  how  to  use  a 
particular  tool  isn't  valid  when  the  focus  is  on  learning  to  use  the  tool  to  enhance  instruction.  These  questions  and 
many  more  will  provide  more  insight  into  learner-friendly  development  of  pre-service  teacher  technology  courses. 
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Abstract 

The  goal  of  this  paper  is  to  demonstrate  how  Constructivism  in  education  has  failed  to  address  criticisms 
by  re-directing  or  misdirecting  the  focus  of  the  debate  over  whether  or  not  Constructivism  is  able  to  give  support  to 
a viable  theory  of  instruction.  In  response,  support  is  given  to  Constructivism  by  drawing  on  rational,  moral,  and 
communicative  frameworks  to  clarify  the  main  Constructivist  tenets  and  to  address  levied  criticisms.  From  this,  the 
recommendation  is  made  that,  depending  on  how  they  are  used,  objective  standards  of  learning  and  evaluation  are 
not  problematic  for  Constructivist  instruction. 

Constructivism 

Constructivism  in  Education  originates  with  the  basic  assumption  that  reality  (or  its  experience)  is  not 
separate  from  but  includes  participants  (regardless  of  the  nature  of  that  participant)  in  its  observations.  The  same 
constructs  and  mutual  meaning  are  construed  as  a gradual  process  of  accommodation  that  achieves  a relative  fit  of 
meaning  constructions. 

Constructivism  concentrates  on  contextually  meaningful  experience  with  the  goal  of  fostering  the 
development  of  autonomy  and  social  reciprocity.  Generally,  Constructivism  is  considered  to  be  a theory  of 
knowledge  and  learning  that  concerns  itself  with  what  one  knows  and  how  it  is  that  one  comes  to  know.  This  theory 
derives  its  views  on  knowledge  from  multiple  sources  (i.e.,  philosophy,  psychology,  anthropology,  sociology,  etc) 
and  considers  knowledge  to  be  internally  constructed  and  socially  mediated.  Learning  is  viewed  as  a self-regulatory 
process  involving  individual  meaning  construction  and  processes  of  social  negotiation  through  social  activity, 
discourse,  and  debate  (Twoney  & Fosnot,  1996). 

According  to  Constructivists,  how  one  constructs  knowledge  is  a function  of  the  prior  experiences,  mental 
structures,  and  beliefs  that  one  uses  to  interpret  objects  and  events.  Constructivism  does  not  preclude  the  existence 
of  external  reality,  it  merely  claims  that  each  of  us  constructs  our  own  reality  of  the  external  world.  Thinking  is 
grounded  in  perception  of  physical  and  social  experiences,  which  can  only  be  comprehended  by  the  mind  (Jonassen, 
1991). 

There  have  been  many  branches  of  Constructivist  theories  put  forth.  In  contemporary  Constructivist 
theory,  the  strongest  individualistic  theory  of  Constructivism  comes  from  von  Glaserfeld  (von  Glaserfeld,  1995; 
1996).  Von  Glaserfeld's  "Radical  Constructivism"  rejects  the  Objectivist  notion  that  knowledge  can  be  treated  as  an 
accurate  representation  of  external  things.  In  contrast  the  author  emphasizes  that  knowledge  be  treated  as  an 
individual  'mapping  of  actions  and  conceptual  operations  that  had  proven  viable  in  the  knowing  subject's  experience' 
(von  Glaserfeld,  1996).  Under  this  view,  no  two  people  produce  the  same  constructs  and  mutual  meaning  is 
construed  as  a gradual  process  of  accommodation  that  achieves  a relative  fit  of  meaning  constructions. 

Spiro,  R.,  Feltovitch,  P.,  Jacobson,  M.,  & Coulson,  R.  (1991b)  offer  a very  selective  version  of 
individualistic  Constructivist  instruction  drawing  form  their  Cognitive  Flexibility  Theory  (Spiro  et  al.,  1987,  1988). 
This  theory  attempted  to  explain  learning  in  ill-structured  domains,  highlighting  the  centrality  of  multiple  knowledge 
representations: 

"Our  Constructivist  position,  as  it  applies  to  complex  and  ill-structured  domains,  rejects  any  view  that  says 
either  that  there  is  no  objective  reality  , or  that  there  is  an  objective  reality  that  can  be  captured  in  any 
single  and  absolute  way.  Rather,  one  of  our  principle  tenets  is  that  the  phenomenon  of  ill-structured  are 
best  thought  of  as  evincing  multiple  truths:  single  perspectives  are  not  false  , they  are  inadequate.  That  is 
why  multiple  knowledge  representations  are  so  central  to  Cognitive  Flexibility  Theory." 

Spiro  et  al.  (1992)  refer  to  their  general  approach  as  'Cognitive  Flexibility  Theory,"  taking  it  to  be  an 
integrated  theory  of  learning,  mental  representation,  and  instruction  that  focuses  on  the  acquisition  of  knowledge  in 
ill-structured  domains.  This  approach  can  be  characterized  as  being  critical  (addressing  deficiencies  in  learning)  and 
involving  multiple  perspectives  or  representations  of  knowledge  (multiple  juxtapositions  of  instructional  content). 


228 

235 


A second  branch  of  contemporary  Constructivism  concentrates  on  the  socially  and  culturally  situated  nature 
of  learning  activity,  drawing  much  of  its  theoretical  basis  from  Vygotsky  (1970;  1979),  social  activity  theorists  ( 
Bordieu,  1976;  Garrison,  1998:  Lave,  1998),  Mead,  and  pragmatism  (Rorty,  1978;  Putnam,  1987).  Garrisons  (1996), 
Meadean  approach  to  Constructivist  epistemology  emphasizes  individual's  self-realization  being  derived  from 
actions  in  the  social  world.  This  view  of  Constructivism  is  largely  embedded  in  a social  context  characterized  by 
argument,  discussion  and  debate.  This  is  described  by  Cunningham  (1991): 

"At  the  heart  of  Constructivism  is  the  notion  that  knowledge  is  constructed,  which  in  the  present  instance 
means  that  our  theoretical  views  are  personal  creations,  embedded  in  a social  context,  within  a social  community 
that  accepts  the  assumptions  underlying  the  perspective.  There  is  no  right  or  wrong  her  in  any  absolute  sense. 
Holding  a theoretical  perspective  means,  making  a personal  commitment  to  it,  while  recognizing  the  potential 
validity  of  other  positions." 

There  are  also  approaches  that  attempt  to  combine  both  individualistic  and  social  approaches  to 
Constructivism.  CTGV  (1991)  view  of  Constructivism  can  be  described  as  combining  elements  of  a individual  and 
social  learning  perspective  in  an  emphasis  on  the  social  nature  of  cognition.  Under  this  view,  the  individual  is  free  to 
build  his  or  her  own  interpretations  of  the  world,  so  long  as  the  interpretation  is  coherent  with  the  general  Zeitgeist. 
Knowledge  is  taken  to  be  a dialectic  process  the  essence  of  which  is  that  individuals  have  opportunities  to  test  their 
constructed  ideas  on  others,  persuade  others  of  the  virtues  of  their  thinking  and  be  persuaded,  (CTGV,  1991). 

Regardless  of  the  Constructivist  theory  supported,  there  are  certain  fundamental  characteristics  of 
Constructivism  that  can  be  enumerated.  First,  there  is  a general  agreement  on  Constructivists'  rejection  of  the 
Objectivist  framework,  which  assumes  that  there  are  established  standards  of  teaching  and  evaluation  that  are  known 
and  can  be  imposed  to  control  learning  and  assess  learning  performance.  Constructivists  oppose  objectivity  on  two 
main  points:  (1)  they  oppose  the  external  control  of  learning  that  is  imposed  independent  of  learning,  and  (2)  they 
oppose  Objectives'  justification  of  objective  standards  of  evaluation  based  on  claims  of  having  objective 
knowledge  of  the  real  world.  However,  this  does  not  mean  that  all  forms  of  objectivity  or  objective  standards  of 
evaluation  have  to  be  rejected  (as  will  be  shown). 

Second,  there  is  a perceived  necessary  link  between  Constructivist  instructional  theory  and  practice.  In, 
"Continuing  the  dialogue",  Duffy  and  Jonassen  (1991)  hold  that  behind  every  good  instructional  design  model  there 
is  a theory  drawn  from  experience.  They  state  that  , "The  models  derived  from  those  experiences  do  not  simply 
reflect  instructional  strategies  and  methods— simple  behavioral  activities.  They  also  reflect  an  underlying 
conceptualization  of  what  it  means  to  learn  and  to  understand."  What  this  does  is  establish  a criterion  that  prevents 
any  external  control  of  learning,  while  at  the  same  time  not  exclude  theoretical  concerns  needed  for  there  to  be  an 
instructional  theory  at  all. 

Many  Constructivist  authors  believe  that  theories  of  learning  and  prescriptions  for  practice  go  hand  in  hand. 
Duffy  and  Jonassen  (1991)  state,  "While  instructional  designers  typically  may  not  have  the  time  or  support  to 
explicitly  apply  a theory  of  learning  during  a design  or  development  task,  the  theory  is  nonetheless  an  integral  part 
of  the  instruction  that  is  produced."  Thus,  learning  cannot  be  assessed  on  the  basis  of  immediate  task  performance. 
Some  aspects  of  learning  are  not  required  until  much  later.  It  is  largely  for  this  reason  that  there  is  an  emphasis  on 
higher-order  learning. 

It  is  also  for  this  reason  that  there  is  an  emphasis  that  learning  environments  should  have  real-world 
relevance  and  that  students  should  be  equiped  with  appropriate  levels  of  complexity  (authentic  tasks).  Learners 
should  actively  engage  in  building  complex  knowledge  structures  and  this  requires  higher-order  thinking 
(knowledge  construction).  Evaluations  need  to  be  able  to  assess  higher  order  thinking  well  (Gagne,  1987;  Merrill 
1983).  It  is  seen  as  more  important  to  evaluate  how  learners  construct  knowledge  (knowledge  acquisition),  rather 
than  to  evaluate  the  product  (process  oriented). 

Third,  there  is  a strong  prescriptive  component  that  marks  a defining  feature  of  the  Constructivist  approach 
(Bednar,  Duffy,  Cunningham,  & Jonassen,  1991).  This  is  reflected  in  how  learning  and  learning  assessments  are 
considered  to  be  a continuous,  ongoing  process,  rather  than  being  tied  to  a specific  task  performance  (Jonassen, 
1991,  1992,  1995).  Also,  Constructivist  instruction  is  directed  towards  the  cultivation  and  assessment  of  higher- 
level  learning  (Spiro,  Feltovitch,  Jacobson,  & Coulson,  1991,  1992).  The  prescriptive  aspect  of  Constructivist 
instruction  is  valuable  because  it  includes  within  its  approach  the  assumption  that  learning  and  instruction  are 
evolving  processes.  This  is  highly  supported  by  those  educational  scholars  who  believe  in  the  importance  of 
educational  ideals  (Searle,  1995;  Steffe,  1995). 

Objectivist  and  Constructivist  responses  to  paridigmatic  criticism 

One  pivotal  methodological  issue  for  Constructivist  theory  surrounds  the  question  of  how  objective 
knowledge  is  tto  be  treated.  Given  that,  terms  like  "objectivist"  and  "objectivity"  have  been  used  in  a multiplicity  of 
ways,  it  must  be  clarified  how  these  terms  are  been  employed  in  the  present  work. 
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The  present  discussion  utilizes  the  general  notion  of  objectivity  as  the  objective  view  of  knowledge  derived 
from  general  philosophy.  The  objective  view  of  knowledge  asserts  that  there  is  a real  and  structured  world  for 
which  one  can  possess  reliable  knowledge  (Putnam,  1994;  Reigeluth,  1991).  Knowledge  obtained  from  the  world 
is  taken  as  stable  and  reliable  because  properties  of  objects  in  the  world  from  which  knowledge  is  obtained  are 
assumed  to  be  relatively  unchanging.  Because  knowledge  is  objective,  the  meaningfulness  of  knowledge  is 
external  to  individuals  and  therefore  can  be  analyzed  and  standardized. 

The  first  criticism  from  Objectivists  opposes  Constructivist’s  failure  to  take  pre-existing  skills  and 
individual  performance  outcomes  into  consideration,  based  on  their  evaluation  of  success  being  represented  by  the 
completion  of  the  learning  task  (Dick,  1991).  Jonassen  (1995)  discusses  the  implications  of  Constructivism  for 
learning  and  instruction: 

’’The  principles  by  which  those  learning  environments  may  be  built  focus  on  four  general  system  attributes: 
context,  construction,  collaboration,  and  conversation.  Constructivist  environments  engage  learners  in 
knowledge  construction  through  collaborative  activities  that  embed  learning  in  a meaningful  context  and 
through  reflection  on  what  has  been  conversation  with  other  learners  ." 

Many  of  the  fundamental  attributes  considered  by  Constructivist  instruction  do  not  start  in  learning  that 
begins  in  the  discrete  learning  situation  to  be  considered.  Rather,  they  have  been  built  from  previous  experience 
(Dick,  1991;  Reigeluth,  1991).  According  to  Objectivists,  such  attributes  as  pre-existing  knowledge  cannot  be 
dismissed.  They  are  a part  of,  and  presupposed  by,  any  Constructivist  instruction.  There  exists  some  standards  due 
to  previous  experience,  without  which,  new  learning  would  be  inconceivable  (Searle,  1981,  1992).  Such  standards 
are  brought  into  new  learning  situations  in  the  form  of  previous  knowledge  and  skills  that  have  been  built  through 
past  experience.  In  the  Objectivist  view,  ignoring  this  is  a serious  oversight  that  plagues  Constructivism  in 
education. 

A second  criticism  from  Objectivists  concerns  the  Constructivist  challenge  of  assessing  learning.  If 
Constructivists  do  not  use  performance  as  an  assessment  of  learning,  then  how  does  one  know  if  learning  has  taken 
place?  Some  Constructivists  posit  that  learning  is  an  emergent  property,  invoking  both  instructors  and  participants, 
that  "falls  out”  of  the  learning  process  (Bednar,  Cunningham,  Duffy,  & Perry,  1991;  Cobb,  1994;  Cobb  & Yackel, 
1996;  Jonassen,  1991).  Cobb  (1996)  argues  that  researchers  have  to  be  included  as  well  due  to  their  interpretive  role 
and  perspective  in  the  educational  context.  Cobb  further  states  that  it  is  social  reality  that  should  dictate  the 
theoretical  perspective.  The  author  draws  on  previous  classroom  research  (Cobb  & Yackel,  1994)  that  utilized 
ongoing  student-instructor  interactions  in  conjunction  with  an  inquiry-based  instruction  and  that  yielded  positive 
support  for  learning  mathematics.  Emergent  learning  explanations,  similar  in  nature  to  Cobb's  (1996)  are  becoming 
increasingly  popular  as  well  (Bereiter,  1994;  Martin  & Sugarman,  1996;  Perkins,  1993;  Prawat,  1996). 

For  some,  these  types  of  learning  explanations  are  not  sufficient.  Reigeluth  (1991)  supports  there  being  a 
need  for  some  sort  of  objectivity  in  contextual  performative  evaluation.  He  accounts  for  the  fundamental  difference 
in  approach  to  learning  assessment  by  the  fact  that  these  other  authors  chose  to  connect  instructional  theory  with 
learning  theory,  whereas  he  treats  them  as  different  and  maintains  the  need  for  there  to  be  a separation. 

Unfortunately,  the  Constructivists'  responses  do  not  sufficiently  address  the  above  levied  criticisms.  Most 
responses  do  not  acknowledge  the  specific  questions,  but  merely  attempt  to  oppose  the  Objectivist  framework 
(Bednar,  Cunningham,  Duffy,  & Perry,  1991,  Jonassen,  1991,  1994,  1997).  Jonassen,  Hennon,  Ondrusek, 
Samouilova,  and  Spaulding  (1997)  provide  opposition  to  the  Objectivist  approach  to  science  by  utilizing  work  from 
hermeneutics  (knowledge  building  through  interpretation),  fuzzy  logic  (evaluating  from  multiple  sources  and 
perspectives  based  on  probabilistic  and  non-linear  nature  of  information),  and  chaos  theory  (emphasizing  non- 
predictable  nature  inherent  in  systems).  Stronger  Constructivist  responses  seek  to  reject  completely  the  criticisms 
levied,  but  more  moderate  Constructivist  responses  seek  to  assimilate  other  perspectives  into  their  own  framework 
(Cunningham,  1991,  1992).  Within  this  view,  objectivity  becomes  translated  into  another  perspective  that  can  be 
subsumed  and  exercised  without  contradicting  other  perspectives.  This  multiple  perspectives  approach 
emphasizes  the  importance  of  integrating  all  views  in  the  evaluation  process  (Jonassen,  1991).  By  not 
acknowledging  specific  questions  and  by  redirecting  the  focus  of  existing  criticisms  made,  Constructivist 
explanations  do  not  address  the  issues,  but  merely  evade  them. 

Towards  an  alternative  view  of  Constructivism 

For  the  most  part,  Constructivist  explanations  share  a common  theoretical  grounding  in  Postmodernist 
philosophy  (Phillips,  1995,  1996;  Prawat,  1996).  Postmodernist  philosophy  can  be  characterized  by  the  breaking  of 
all  ties  with  existing  foundational  modes  of  thought  and  replacing  this  structure  with  a relativistic  philosophy  that 
embraces  multiple  frameworks  of  meaning  found  in  particularistic  accounts,  (Guba  & Lincoln,  1985;  Howe,  1988; 
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Rorty,  1986).  Guba  and  Lincoln  (1985)  believe  there  to  be  no  possibility  of  accomodation  between  the 
foundational  scientific  and  Postmodernist  paradigms  because  the  basic  assumptions  of  the  two  are  in  conflict. 

Adherance  to  a Postmodernist  framework  is  an  implicit  assumption  that  runs  through  numerous 
Constructivist  theories,  such  as  von  Glaserfeld's  (1990)  "radical  Constructivism"  and  the  majority  of  social 
Constructivist  work  (Cunningham,  1991,  1992;  von  Glaserfeld,  1990).  Radical  Constructivism  (1990)  holds  that 
there  is  no  objective  reality  that  can  be  uniformly  interpreted  by  all  , while  Cunningham's  (1991)  Social 
Constructivism  calls  for  the  integration  of  multiple  perspective  taking  to  accommodate  individuals'  differing 
interpretations.  Much  of  the  criticism  of  Constructivism  (Reigeluth,  1991;  Dick,  1991)  is  directed  at  the  lack  of 
objective  standards  and  means  of  rational  legitimization  entailed  by  adopting  Postmodernist  philosophy. 

There  are  alternatives  to  Postmodernist  philosophy  that  allow  for  multiple  frameworks  of  meaning,  which 
characterize  Constructivist  explanations.  Putnam  (1990)  argues  for  the  possibility  of  appealing  to  objective  rational 
standards  by  demonstrating  the  need  for  particular  communities  and  practices  to  rely  on  some  level  of  underlying 
conditions  required  for  evaluative  judgements  to  be  made  in  the  Constructivist  domain  or  any  other  domain.  This 
demonstrates  the  potential  of  Constructivism  to  overcome  certain  limitations  that  have  been  pointed  out  (Dick,  1991; 
Reigeluth,  1991). 

A more  comprehensive  view  of  Constructist  instruction  can  be  obtained  by  examining  the  cognitive  tools 
available  (Kommers,  Jonassen,  & Mayes,  1992)  within  a non-Postmodemist  perspective  (Putnum,  1990;  Siegal, 
1996).  Attempting  to  theoretically  ground  cognitive  and  non-cognitive  tools  relevant  to  the  Constructivist  position 
will  provide  a more  extensive  view  essential  to  providing  any  serious  support  to  the  Constructivist  position. 

Addressing  criticisms  to  Constructivism  by  drawing  on  multiple  frameworks 

This  section  deals  with  criticisms  to  Constructivism  by  presenting  evidence  from  areas  outside  the 
literature.  The  literatures  drawn  from  provide  innovative  ways  to  defend  Constructivism,  referred  to  as  tools.  These 
tools  are  divided  in  cognitive  and  non-cognitive  types.  Cognitive  tools  refer  to  individual  psychological  processes, 
whereas,  non-cognitive  tools  refer  to  broader  normative  acts  such  as  discourse  practices. 

Theoretically  grounding  cognitive  tools  in  support  of  Constructivism 

In  Rationality  Redeemed?,  Siegel  (1996)  introduces  his  theory  as  a continuation  of  longstanding  efforts  to 
demonstrate  that  rationality  possesses  an  "educational  cognate"  in  critical  thinking  (Barnes,  1992),  which  represents 
an  educational  ideal.  This  work  stems  partly  from  an  effort  to  address  anti-Enlightenment  tenets  present  in 
philosophy  of  education.  It  acts  as  both  a response  to  existing  arguments  and  as  a further  development  of  the 
Siegel's  rationalist  thesis. 

The  main  tenet  of  the  thesis  is  that  the  ideal  of  education  is  to  promote  rationality  and  critical  thinking. 
This  rationality  is  broad,  involving  both  epistemological  and  moral  dimensions.  Siegel  (1996)  defends  the  critical 
thinker  as  someone  who  possesses  specific  characteristics: 

"A  critical  thinker  must  have,  then  both  a solid  understanding  of  the  principles  of  reason  assessment,  and 
significant  ability  to  utilize  that  understanding  in  order  to  evaluate  properly  beliefs,  actions,  judgements, 
and  the  reason  which  are  thought  to  support  them.  This  dimension  of  critical  thinking  may  be  called  the 
reason  assessment  component  of  critical  thinking." 

Following  this,  Siegel  outlines  two  principles  of  reason  assessment:  general  (applicable  to  many  domains) 
and  subject-specific  (domain  specific).  Grasping  these  principles  is  considered  to  be  an  important  part  of  critical 
thinking.  Ultimately,  Siegal  supports  that  critical  thinking  is  an  educational  ideal  characterized  by  the  following 
attributes:  respect  for  students  as  persons,  self-sufficiency  and  preparation  for  adulthood,  initiation  into  the  rational 
tradition,  and  democratic  living.  This  educational  ideal  is,  therefore,  also  a moral  ideal.  Thus,  although  critical 
thinking  links  directly  with  epistemology,  it  involves  other  dimensions  as  well  when  it  is  considered  an  educational 
ideal.  It  is  also  for  this  reason  that  Siegal  believes  it  is  important  for  philosophy  and  philosophy  of  education  to 
engage  in  discourse. 

This  broad  notion  of  rationality  has  not  only  an  epistemological  dimension,  but  also  a moral  dimension. 
For  this  reason,  a large  range  of  cognitive  tools  can  be  congruent  with  Constructivism's  prescriptive  component, 
concerned  with  learning  activity  that  includes  individuals'  attitudes  (moral). 

Another  important  cognitive  tool  for  Constructivist  instruction  is  the  imagaination.  Imagination  is  a 
cognitive  tool  that  represents  Constructivist  instruction  as  a higher  order  learning  skill.  Egan  (1992)  provides  a very 
useful  aid  in  shedding  light  on  a powerful  cognitive  tool  that  contributes  greatly  to  learning.  This  is  important  to  the 
present  discussion  on  Constructivism  as  a means  of  pointing  to  cognitive  capacity  often  overlooked  by  instructional 
theories  but  essential  to  Constructivist  instruction.  Egan  incorporates  the  cognitive  tool  of  imagination  into 
educational  practices. 
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Egan  (1992)  argues  for  the  necessity  of  imagination  to  be  incorporated  in  education.  He  attempts  to  carry 
his  concept  of  'imagination'  into  discussions  of  conventional  thinking,  emphasizing  the  need  for  freedom  of  mental 
activity: 

"Imagination  is  what  enables  this  transcendence,  and  is  consequently  necessary  to  education.  It  is 
important  because  transcending  the  conventional  is  necessary  to  constructing  one's  sense  of  any  area  of 
knowledge:  accepting  conventional  representations  is  to  fail  to  make  knowledge  one's  own.  It  is  to  keep  it 
inert  rather  than  incorporate  it  in  one’s  life." 

Egan  remarks  on  the  tension  that  exists  between  conventions  and  imagination,  marking  it  as  a problem  that 
has  to  be  worked  out.  Also,  Egan  distinguishes  the  way  in  which  humans  learn  from  the  ways  computers  operate  in 
order  to  demonstrate  that  learning  is  not  simply  a question  of  recording  symbols  for  later  retrieval: 

"If  we  allow  our  technologies  to  determine  how  we  think  about  our  intellectual  processes,  then  one  effect, 
which  has  been  pervasive  and  very  damaging  to  education,  is  to  think  of  learning  as  a process  analogous  to 
recording  symbols  in  the  mind  for  later  retrieval." 

Here,  Egan  is  attempting  to  draw  attention  to  the  fact  that  the  meaningfulness  of  learning  requires  a more 
flexible  approach.  He  (1992)  states,  "The  more  flexibly  we  can  think  of  things  as  possibly  being  so,  the  richer,  and 
more  unusual  and  effective  can  be  the  meaning  we  compose."  Thus,  memorization  of  knowledge  only  is  simply  not 
enough.  However,  the  memory  is  considered  to  be  important  to  the  imagination,  since  it  is  that  which  is  contained 
in  one’s  memories  that  the  imagination  draws  on  with  which  to  construct. 

The  above  discussion  on  imagination  builds  on  the  discussion  of  cognitive  elements,  giving  attention  to  a 
cognitive  tool  that  has  been  neglected  in  the  literature.  In  addition,  the  discussion  on  imagination  makes  an  appeal  to 
non-relativistic  standards  that  extend  beyond  the  practical  instruction,  transcending  conventions.  This  gives  a 
prescriptive  element  into  educational  practices  that  Constructivist  instruction  encourages. 

Theoretically  grounding  non-cognitive  tools  in  support  of  Constructivism 

Recent  innovative  Constructivist  research  in  the  area  of  moral  and  cooperative  learning  has  contributed  to 
extending  what  is  understood  to  represent  Constructivism's  theoretical  grounding  (DeVries,  Reese-Learned,  & 
Morgan,  1991;  DeVries  & Zan  , 1996).  This  work  has  demonstrated  that  Epistemology  is  an  element  of  the  whole 
of  Constructivist  philosophy,  which  is  dynamic,  including  not  only  descriptive  but  prescriptive  elements.  That  is,  it 
is  not  only  about  what  is  but  also  what  could  be.  This  is  an  important  attribute  of  Constructivism  that  has  been 
largely  neglected. 

The  presence  of  broad  treatments,  such  as  Larochelle  and  Bednarz's  (1998),  suggests  that  the  foundation  of 
Constructivism  cannot  be  exhausted  by  epistemological  legitimization.  There  are  also  needs  for  cooperation  and 
discourse,  which  have  moral  as  well  as  epistemological  conditions.  Sensitivity  to  these  non-cognitive  tools  is 
required  for  Constructivist  instruction  be  initiated  (Jonassen,  1994). 

The  moral  dimension  of  Constructivism  has  received  little  attention  in  the  contemporary  Constructivist 
literature.  However  the  need  to  consider  moral  dimensions  of  Constructivism  is  a pressing  task  (Piaget,  1965; 
1981).  Piaget  (1965)  argued  against  the  Objectivist  approach  to  education  where  instructors  controlled  all  aspects  of 
student  learning.  In  drawing  from  his  own  work,  he  observed  that  such  a control  over  learning  resulted  in  mindless 
moral  and  intellectual  conformity,  leading  to  self-doubt,  lack  of  curiosity,  uncritical  thinking,  and  problems  in 
cooperative  interactions.  DeVries  and  Zan  (1996)  support  a sociomoral  theory  of  Constructivism  aimed  at 
encouraging  cooperative  sociomoral  development.  The  authors  believe  that  such  considerations  are  necessary  in 
order  to  prevent  the  loss  of  individual  knowledge  constructions  and  self-regulatory  learning. 

Attention  is  directed  at  promoting  multiple  perspective  taking  and  moral  reasoning  among  children  to 
encourage  greater  interpersonal  understandings.  This  perspective  draws  on  previous  research  (DeVries,  Reese- 
Learned,  & Morgan,  1991),  which  found  that  children  subjected  to  a Constructivist  classroom  setting  invested 
greater  effort  to  resolve  interpersonal  conflicts  compared  to  children  subjected  to  eclectic  or  didactic  oriented 
kindergarten  classroom  settings.  From  this  work,  Devies  & Zan  (1996)  support  that  rule  and  decision  making 
should  be  directed,  promoting  feelings  of  necessity  and  fairness  in  rule  making,  a sense  of  commitment  and 
ownership  in  decision  making,  and  a sense  of  shared  responsibility  in  how  the  group  gets  along.  The  result  of  this  is 
believed  by  the  authors  to  encourage  mutual  respect  among  learning.  From  this  they  conclude  that  a moral 
classroom  contributes  general  conditions  for  intellectual  development. 

How  can  broadening  Constructivism  to  include  moral  attributes  aid  in  dealing  with  posited  criticisms? 
First,  on  a theoretical  level,  it  contributes  to  the  understanding  of  Constructivist  Instruction  as  not  only  descriptive 
but  prescriptive  as  well.  This  builds  on  the  discussion  of  cognitive  tools  by  tying  the  prescriptive  value  of 
Constructivism  to  actual  Constructivist  research.  On  a practical  level,  it  extends  knowledge  of  Constructivism's 
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breadth  with  respect  to  learning  content.  It  demonstrates  that  Constructivism  is  not  only  limited  to  epistemological 
constructions  but  also  includes  moral  constructions.  More  importantly,  this  work  brings  to  the  surface  an  important 
aspect  of  Constructivism  that  has  never  been  actually  pursued  but  only  speculated  on  by  Bednar,  Cunningham, 
Duffy,  and  Perry  (1991): 

"Optimally,  we  would  tie  our  prescriptions  for  learning  to  a specific  theoretical  position— the  prescriptions 
would  be  a realization  of  a particular  understanding  of  how  people  learn.  Minimally,  we  must  be  aware  of 
the  epistemological  underpinnings  of  our  instructional  design  and  we  must  be  aware  of  the  consequences  of 
that  epistemology  on  our  goals  for  instruction,  our  design  of  instruction,  and  on  the  very  process  of  design.” 

In  addition,  there  is  something  extremely  important  to  recognize  when  assuming  a theoretical-practical 
connection  that  has  not  been  addressed  in  the  Constructivist  literature.  This  is  perhaps  one  of  the  most  powerful 
Constructivist  tools  that  has  yet  to  be  discovered.  Given  that  there  is  an  assumed  theoretical-practical  connection  at 
the  heart  of  all  Constructivist  inquiry  (Bednar,  1991;  Cunningham,  1991;  Jonassen,  1991),  any  extension  of 
Constructivist  practices  (e.g.,  moral  and  cooperative  learning)  also  assumes  a theoretical  connection  beyond  other 
knowledge  constructions  acquired  in  experience.  In  this  way,  the  theory  of  Constructivism  is  itself  a meta- 
construction or  meta-cognition.  This  is  similar  to  Siegal's  (1996)  argument  against  Postmodernism. 

One  way  to  bind  these  cooperative  and  communicative  practices  to  a theoretical  grounding  that  supports 
both  morality  and  rationality  has  been  posed  by  Habermas  (1990;  1993).  Habermas  advocates  a communicative 
theory  of  meaning  where  claims  of  validity  and  truth  are  decided  by  resolving  normative  rightness,  which  can  be 
determined  through  discursive  argumentation.  He  summarizes  the  generalized  moral  imperative  that  corresponds  to 
his  theory  of  argumentative  discourse.  He  states  that,  "All  affected  can  accept  the  consequences  and  the  side  effects 
its  general  observance  can  be  anticipated  to  have  for  the  satisfaction  of  everyone's  interests  (and  these  consequences 
are  preferred  to  those  of  known  alternative  possibilities  for  regulation)." 

Habermas  makes  a concerted  effort  to  bridge  the  gap  between  appeals  for  the  communal  shaping  of 
values/practices  with  the  autonomous  role  of  the  rational  individual.  For  Habermas,  moral  practices  are  social 
matters  to  be  decided  by  discourse  interactions  of  individually  deliberating  subjects.  Thus,  both  individual  will  and 
community  practices  are  taken  into  consideration  by  Habermas'  (1990)  universal  theory  of  argumentative  discourse. 

The  importance  this  has  for  Constructivism  is  threefold:  (1)  it  provides  a means  of  making  a theoretical- 
practical  bridging  generally  assumed  within  the  Constructivist  domain,  (2)  it  provides  a connection  between 
rationality  and  morality,  and  (3)  it  provides  a means  of  connecting  the  individual  and  collective  group. 

Habermas’  work  could  be  used  to  support  the  Constructivist  Theory  of  Learning  and  Instruction  against 
Objectivist  criticisms.  Much  of  the  criticisms  surrounding  Constructivism  are  directed  at  the  inability  to  have  any 
objective  standards  (Reigeluth,  1991).  However,  the  theoretical  grounding  provided  above  with  Habermas’  Kantian 
project  (1990)  offers  certain  objective  standards.  Kant  (1787)  believed  that  the  only  genuine  morality  is  one  that 
would  be  objectively  and  universally  binding.  It  would  apply  to  all  people  and  be  the  same  for  all.  The  basis  of  this 
morality  that  applied  to  all  resided  in  rational  nature.  For  Kant,  it  was  rational  nature  that  provided  the  binding  force 
in  which  to  ground  morality.  In  his  "Critique  of  Practical  Reason”  (1787),  Kant  defines  "practical  principles"  and 
discusses  their  moral  implications.  Kant  also  states  that,  "Practical  principles  are  propositions  which  contain  a 
general  determination  of  the  will,  having  under  it  several  practical  rules.  They  are  subjective,  or  practical  laws, 
when  the  condition  is  recognized  as  objective,  i.e.,  as  valid  for  the  will  of  every  rational  being." 

This  notion  of  objectivity  is  not  the  same  as  that  to  which  Constructivism  is  opposed.  The  brand  of 
objectivity  that  Constructivism  opposes  claims  that  knowledge  representations  correspond  to  the  external  world  and 
that  externally  imposed  standards  based  on  this  knowledge  can  be  used  to  control  teaching.  This  brand  of 
objectivity  controls  the  learner  and  prevents  learning  to  be  constructed.  This  is  very  different  from  the  brand  of 
objectivity  discussed  by  Kant  (1781),  which  does  not  claim  objective  knowledge  of  the  external  world  at  all. 

Instead  it  assumes  only  objectivity  that  corresponds  to  the  individuals'  shared  rational  nature.  It  is  not  an 
externally  imposed  standard  to  control  learning,  but  rather,  represents  an  integral  characteristic  of  each  individual 
to  be  constructed  in  practical  experience  with  others.  What  makes  this  limited  brand  of  objectivity  so  attractive  is 
that  it  allows  for  objective  standards  to  be  pursued  as  an  essential  part  Constructivist  Instruction.  At  the  same  time, 
it  does  not  enforce  any  objective  standards  that  would  control  learning.  This  is  overcome  by  viewing  objective 
standards  as  prescriptive  standards  to  be  progressed  toward  but  never  fully  obtained.  This  form  of  objectivity  as  a 
rational  standard  is  substantiated  by  the  efforts  throughout  the  paper  to  demonstrate  the  prescriptive  components  of 
Constructivist  Instruction  entailed  by  Constructivism’s  theoretical-practical  assumption. 

How  does  this  contribute  to  the  problem  of  learning  assessments?  How  does  an  objective  rational 
grounding  address  standards  of  evaluation  required  for  effective  instruction?  The  contribution  made  lies  in  how  it 
is  that  standardized  evaluative  measures  essential  to  instruction  are  treated.  Under  this  view,  problems  of 
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evaluation  are  resolved  by  Constructivist  instructions'  prescriptive  function.  First,  evaluations  would  not  simply  be 
administered  but  would  be  integrated  as  part  of  the  learning  process.  This  could  be  accomplished  by  making  clear 
who  is  responsible  for  creating  the  evaluative  standards  and  when.  This  way  students  can  feel  they  are  not  merely 
subjected  to  some  imposed  standard,  but  rather  are  participating  in  the  standard  evaluation.  This  is  done  so  that 
students  can  learn  to  understand  the  standard  as  a first  step  in  being  able  to  participate  in  the  evaluation  and 
selection  of  future  standards.  This  can  be  taken  to  be  a type  of  cognitive  apprenticeship  (Clancy,  1992;  Cobb, 
1996;  Collins,  1991). 

Second,  learners  are  participating  in  standard  evaluations  administered  not  with  the  understanding  that  the 
standard  is  objectively  true  but  rather,  is  a logical  possibility,  objectively  true  for  all  learning  participants  and  to  be 
worked  towards  in  a cooperative  manner  (Habermas,  1995;  Kagan,  1990).  This  captures  the  essence  of  what 
Constructivism  should  encourage  when  attempting  to  provide  instruction  in  an  educational  setting. 

Conclusion 

The  previous  discussion  yields  positive  support  for  the  possibility  of  supporting  a Constructivist  Theory  of 
Learning  and  Instruction  without  adopting  a Postmodernist  stance.  The  following  conclusions  are  drawn:  (1)  a close 
theoretical-practical  connection  assumed  to  be  a defining  element  of  Constructivist  Theory  and  can  be  given  support 
independent  of  a Postmodernist  framework  by  appealing  to  a broader  notion  of  rationality  that  is  objectively 
grounded  and  co-extensive  with  the  prescriptive  idealization  of  education,  and  (2)  a theoretical  exploration  of 
practical  cognitive  and  non-cognitive  tools  reveals  a connection  between  these  tools  with  prescriptive  elements 
valuable  to  Constuctivism  and  evolving  education. 

The  issue  of  objective  standards  within  Constructivism  is  found  not  to  be  a problem  as  many  critics 
purport.  While  Constructivist  Instruction  does  not  base  itself  on  the  thesis  of  objective  realism,  incorporating 
objective  standards  within  Constructivist  instruction  without  opposing  its  fundamental  tenets  is  a challenge. 
Concentrating  on  the  necessity  of  rationality  and  the  prescriptive  component  of  Constructivist  Instruction  have 
revealed  promising  objective  standards  in  which  to  imbed  Constructivism.  This  adds  an  important  element  to  the 
existing  Constructivist  paradigm. 

Supporting  the  necessity  of  rationality  and  the  prescriptive  component  of  education  has  revealed  promising 
objective  standards  in  which  to  imbed  Constructivist  Instruction.  Future  work  should  be  concerned  with 
determining  the  extent  to  which  such  objective  standards  are  effective  for  Constructivist  instruction. 
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Abstract 

This  paper  provides  a discussion  of  the  theoretical  and  methodological  implications  of  designing  a problem-based 
professional  development  system  on  the  Web  as  well  as  describes  an  instructional  design  model  using  problem 
based  learning  (PBL)  principles.  The  purpose  of  the  paper  is  to  help  instructional  designers  understand  the  methods 
used  by  the  LTTS  team  in  developing  and  structuring  PBL  on  the  Web. 


Background 

Telecommunication  networks  are  changing  the  nature  of  teaching  and  learning.  In  the  past  decade,  Web- 
based  learning  has  experienced  rapid  growth  in  various  educational  arenas.  From  corporate  training  to  K-12  and 
higher  education,  many  educational  providers  are  now  providing  online  courses  and  learning  modules  so  learners 
can  gain  access  to  education  anytime  and  anywhere  there  is  an  Internet  connection. 

Currently,  more  than  50,000  university  courses  are  taught  online,  and  1,000  universities  developing  and 
offering  these  online  courses  (Camavale,  2000;  National  Center  for  Education  Statistics,  1999).  Nearly  all  of 
Fortune  100  companies  already  offer  some  form  of  online  computer-based  training  (Herther,  1997),  and  this  will 
continue  to  increase  as  the  demand  for  stand  up  training  decreases  rapidly  (ASTD,  2000). 

A need  for  periodic  professional  development  has  contributed  to  the  demand  for  more  flexible  access  to 
higher  education.  Organizations  and  institutions  are  increasingly  offering  online  professional  development 
opportunities  to  educators  (Mather,  2000,  Schrum,  1999).  In  current  times,  continual  reskilling  is  a fact  of  life 
(Bonk  and  Wisher,  2000),  especially  for  teachers  who  must  keep  up  with  new  teaching  strategies,  the  latest 
professional  standards,  and  constantly  changing  technologies.  Coincidentally,  new  technologies  promise  to  facilitate 
access  to  learning  at  times  and  places  chosen  by  the  learner  (Albion  and  Gibson,  1998).  These  web-based 
environments  have  the  potential  to  transform  teacher  professional  development  through  the  use  of  new  models  of 
teaching  and  learning.  Online  environments  also  have  the  potential  to  facilitate  a sustained  culture  of  sharing, 
collaboration,  mentoring,  and  support  for  K-12  teachers. 

With  the  rapid  rate  of  expansion  of  online  education,  there  has  also  been  a call  for  a renewed  focus  on 
understanding  and  improving  online  teaching  and  learning.  As  a result,  institutions  are  designing  and  implementing 
new  models  of  distance  learning  environments  (Institute  for  Higher  Education,  1999).  The  goal  of  these  learning 
environments  is  to  promote  learner  engagement  using  inquiry  and  problem  solving.  One  such  methodology  being 
implemented  online  is  problem-based  learning.  Problem-based  learning  (PBL)  is  often  promoted  in  response  to  the 
current  need  to  offer  authentic  and  effective  professional  education.  Jonassen  (1991)  argues  that  "the  most  effective 
learning  contexts  are  those  which  are  problem  or  case  based  and  activity  oriented,  that  immerse  the  learner  in  the 
situation  requiring  him  or  her  to  acquire  skills  or  knowledge  in  order  to  solve  the  problem  or  manipulate  the 
solution"  (p.  36). 

Problem-based  learning  is  a curriculum  development  and  instructional  system  that  simultaneously  develops 
both  problem  solving  strategies  and  disciplinary  knowledge  bases  and  skills  by  placing  students  in  the  active  role  of 
problem  solvers  confronted  with  an  ill-structured  problem  that  mirrors  real-world  problems  (Finkle  and  Torp,  1995). 
Traditionally,  PBL  is  used  in  face-to-face  environments,  with  a facilitator  guiding  collaborative  teams  of  students  in 
solving  a problem.  PBL  was  initially  developed  at  McMasters  University  in  the  late  1960s.  It  is  used  in  a wide 
variety  of  educational  environments  including  medical  education  (Barrows,  1985),  business  administration  (Stinson 
and  Milter,  1996),  schools  of  education  (Bridges  and  Hallinger,1992),  undergraduate  education  (White,  1996),  and  K- 
12  schools  (Barrows  and  Myers,  1993).  Problem  based  learning  environments  are  often  reported  to  increase  student 
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motivation,  to  help  develop  critical  thinking  skills,  to  increase  use  of  outside  learning  resources,  and  to  increase 
understanding  of  content  knowledge  in  context  of  its  use  (Albanese  and  Mitchell,  1993,  Torp  and  Sage,  1997). 

As  traditional  PBL  has  naturally  expanded  to  many  curricular  areas,  it  has  also  expanded  to  the  online 
educational  arena  (Stinson  and  Milter,  1996;  Oliver  and  Omari,  1999;  Pankratz,  1998;  Naidu  and  Oliver,  1996).  Yet 
online  learning  environments  are  new,  and  principles  for  the  design  of  instruction  in  this  environment  are  just 
emerging  [Duffy,  Dueber,  and  Hawley,  1999;  Bonk  et  al,  in  press].  This  means  that  instructional  designers  are  the 
first  who  face  the  challenge  of  incorporating  PBL  approaches  into  the  Web  structure. 

Heath  (1997)  recognizes  a trend  in  instructional  design  towards  replacing  traditional  behaviorist  approaches 
with  constructivist  orientations  emphasizing  the  use  of  emerging  technologies.  Therefore,  understanding  how  to 
design  and  support  the  online  problem  based  learning  process  is  critical  to  the  effectiveness  of  such  online  learning 
systems.  Since  few  instructional  designers  have  experience  in  developing  problem  based  instruction  for  a Web  based 
learning  environment,  it  is  critical  that  we  provide  instructional  models  focusing  on  the  design  of  online  problem 
based  learning  systems. 

In  this  paper,  we  provide  an  instructional  design  model  for  designing  a problem-based  professional 
development  system  on  the  Web,  called  the  Learning  to  Teach  with  Technology  Studio.  This  model  will  help 
instructional  designers  better  understand  the  theory  and  methodology  of  online  problem  based  learning  and  enable 
them  to  adapt  it  as  needed  for  their  own  online  learning  environments.  This  model  will  also  help  support  new 
models  of  professional  development  for  K-12  teachers.  Understanding  how  to  design  online  professional 
development  systems  for  in-service  teachers  is  important  given  recent  recommendations  from  the  professional 
development  literature.  The  design  of  these  online  learning  environments  presents  unique  opportunities  for 
transforming  current  models  of  preK-12  professional  development.  “Rather  than  having  information  delivered  to 
them,  teachers  need  to  examine  their  beliefs  about  subject  matter,  student  learning  and  instruction  in  the  light  of 
innovation"  (Marx,  Blumenfeld,  Krajcik,  and  Soloway,  1998,  p.  33).  These  models  build  on  continuous  inquiry, 
integration  of  new  ideas  with  colleagues,  and  reflective  practice,  which  are  critical  elements  of  successful 
professional  development  (Shanker,  1990). 

Theoretical  Framework 

Before  we  discuss  the  instructional  model  used  in  the  design  of  the  Learning  to  Teach  with  Technology 
Studio,  it  is  important  to  address  the  theoretical  framework  that  was  used  as  the  basis  for  its  development. 
Constructivism  served  as  the  guiding  theoretical  framework  for  the  development  of  this  instructional  model. 
Constructivist  theories  of  learning  posit  that  knowledge  evolves  through  social  negotiation  and  through  the  viability 
of  individual  understandings,  that  understanding  come  from  our  interactions  with  the  environment,  and  that 
cognitive  conflict  or  puzzlement  is  the  stimulus  for  learning  and  determined  the  nature  of  what  is  learned  (Duffy  and 
Cunningham,  1 996;  Saveryand  Duffy,  1996). 

During  the  past  several  decades,  there  has  been  an  important  shift  in  the  way  we  view  knowledge  as  well  as 
the  process  of  learning.  Knowledge  was  once  viewed  as  a known  set  of  discrete  facts  to  be  acquired  during  the 
learning  process  via  a simple  process  of  inputting  information  into  a learner  ‘s  head.  Within  a Constructivist 
framework,  the  focus  is  on  the  role  of  the  learner.  Rather  than  simply  acquiring  existing  knowledge,  the  learner 
constructs  knowledge  through  a complex  set  of  interactions  with  the  environment,  culture,  negotiations  with  other 
people,  and  tools  (technological  and  otherwise)  used  in  the  process  of  learning.  As  learners  engage  in  the  social 
construction  of  knowledge,  meaning,  practice,  and  context  are  inextricably  woven  together  (Lave  and  Wenger, 
1991;  Naidu  and  Oliver,  1996).  Savery  and  Duffy  (1996)  link  the  theoretical  principles  of  constructivism  with  the 
methodology  of  problem  based  learning  as  follows: 

• Learning  should  be  relevant 

• Instructional  goals  should  be  consistent  with  the  learner’s  goals. 

• Cognitive  demands  and  tasks  in  the  learning  environment  should  be  consistent  with  cognitive  demands  and 

tasks  for  the  environment  for  which  the  learner  is  being  prepared. 

• Teachers'  role  is  to  challenge  the  students'  thinking. 

• Students'  ideas  should  be  tested  against  alternate  views  through  social  negotiation  and  collaborative 

learning  groups. 

• Encourage  reflection  on  the  learning  process  (p.  1 37). 

By  using  Constructivist  theories  of  learning  as  the  basis  for  the  development  of  this  instructional  model,  we 
considered  the  role  of  the  learner,  the  knowledge  construction  process,  and  the  learning  environment.  By  using  the 
principles  of  Constructivism  to  guide  the  development  of  an  online  problem  based  learning  instructional  model,  we 
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can  design  educational  environments  that  further  develop  learners’  critical  thinking  and  problem  solving  abilities, 
content  knowledge,  skills,  strategies,  and  learning  processes. 

The  Learning  to  Teach  with  Technology  Studio 

In  order  to  understand  this  instructional  model,  it  is  first  necessary  to  understand  the  context  in  which  this 
instructional  model  was  created.  In  1999,  Indiana  University’s  Center  for  Research  on  Learning  and  Technology 
received  a grant  from  the  Department  of  Education  to  develop  the  Learning  to  Teach  with  Technology  Studio 
(LTTS).  The  LTTS  is  a web-based  professional  development  system  to  help  K-12  teachers  learn  to  use  technology 
to  support  student  inquiry  and  problem  solving.  (See  http://ltts.org  for  more  information).  The  LTTS  is  being 

developed  to  address  needs  of  K-12  inservice  teachers  who  lack  skills  and  confidence  in  integrating  technology  into 

their  teaching.  While  the  technology  infrastructure  grows  — 51%  of  classrooms  are  wired  for  Internet  access,  and 
there  is  one  computer  for  every  5-7  students  (National  Center  for  Education  Statistics,  1999;  President’s  Panel, 
1997)  — the  ability  to  use  it  lags.  The  Milken  Exchange  (Solmon,1998)  found  that  teachers  do  not  model  the  use  of 
IT  skills  in  their  teaching.,  Eighty  percent  of  teachers  report  that  they  do  not  feel  well  prepared  to  integrate 
technology  with  their  teaching  (National  Center  for  Education  Statistics,  1999).  Little  seems  to  have  changed  since 
the  1995  OTA  report  concluded,  “Overall,  teacher  education  programs  in  the  U.S.  do  not  prepare  graduates  to  use 
technology  as  a teaching  tool”  (Office  of  Technology  Assessment,  1995,  p.  184). 

The  challenges  of  this  were  to  design  an  online  learning  environment  based  on  the  following  criteria: 

• To  meet  the  needs  of  K-12  teachers  in  helping  them  learn  about  technology  integration  issues 

• To  create  a learning  anytime  and  anywhere  which  is  open  entry  and  open  exit 

• To  design  a learning  environment  that  could  be  used  by  teachers’  current  technology  levels  as  well  as 
technological  resources,  which  are  all  extremely  varied 

• To  design  a learning  environment  that  emphasizes  the  individual  but  promotes  community 

• To  design  a learning  environment  that  integrates  the  latest  research  and  pedagogical  innovations  into 
daily  classroom  practice 

• To  provide  high  quality  resources  for  learning 

• To  enable  teachers  to  enhance  their  knowledge  for  using  technology  in  their  subject  area  while 

addressing  professional  standards 

The  current  emphasis  on  technology  is  to  ensure  that  it  is  used  effectively  to  create  new  opportunities  for 
learning  and  to  promote  student  achievement.  Educational  technology  requires  the  assistance  of  educators  who 
integrate  technology  into  the  curriculum,  align  it  with  student  learning  goals,  and  use  it  for  engaged  learning 

projects”  (NCREL,  2000)  "Teacher  quality  is  the  factor  that  matters  most  for  student  learning,"  note  Darling- 

Hammond  and  Berry  (1998).  Therefore,  professional  development  for  teachers  becomes  the  key  issue  in  using 
technology  to  improve  the  quality  of  learning  in  the  classroom. 

To  address  this  need,  the  LTTS  is  being  developed  to  provide  learning  modules  that  help  teachers  learn  to 
integrate  technology  to  support  student  inquiry  and  problem  solving.  These  learning  modules  are  self-contained, 
problem-based  learning  packages  where  a learner  is  presented  with  a problem  scenarios  based  on  significant 
technology  integration  issues  that  they  face  today,  such  as  learning  how  to  choose  Internet-based  projects,  design 
WebQuests,  and  evaluate  information  found  on  the  Internet.  The  goal  for  solving  the  problem  is  related  to  the 
teachers’  own  classroom  context.  So  the  teacher  may  develop  an  Internet-based  teaching  unit  for  her  own  class, 
choose  an  appropriate  technology  for  her  own  students’  collaboration,  or  learn  how  to  teach  Internet  search  models 
to  her  students. 

Considering  the  meaning  of  professional  development  in  the  technological  age,  Grant  states: 

"Professional  development  goes  beyond  the  term  'training'  with  its  implications  of  learning  skills, 
and  encompasses  a definition  that  includes  formal  and  informal  means  of  helping  teachers  not  only 
learn  new  skills  but  also  develop  new  insights  into  pedagogy  and  their  own  practice,  and  explore 
new  or  advanced  understandings  of  content  and  resources.  [This]  definition  of  professional 
development  includes  support  for  teachers  as  they  encounter  the  challenges  that  come  with  putting 
into  practice  their  evolving  understandings  about  the  use  of  technology  to  support  inquiry-based 
learning”  (NCREL,  2000). 

Additionally,  the  U.S.  Department  of  Education  (1995)  set  forth  several  related  principles  that  professional 
development  should  meet,  including: 


• It  should  reflect  the  best  available  research  and  practice  in  teaching,  learning,  and  leadership. 
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• It  should  enable  teachers  to  develop  further  expertise  in  subject  content,  teaching  strategies,  uses  of 
technologies,  and  other  essential  elements  in  teaching  to  high  standards. 

• It  should  promote  continuous  inquiry  and  improvement  embedded  in  the  daily  life  of  schools. 

In  developing  the  LTTS,  these  professional  development  principles  were  considered  as  part  of  the  design. 
Problem  based  learning  was  chosen  as  the  methodology  because  of  the  strengths  and  advantages  it  offers  for 
supporting  teachers’  professional  development  and  learning.  First,  PBL  builds  on  the  use  of  teachers’  research  and 
practice  in  teaching  and  learning.  Because  the  learner  must  address  real  classroom  problems  and  issues  using  the 
latest  research  and  other  resources,  they  are  connecting  research  to  classroom  teaching  and  practice. 

Third,  PBL  provides  a model  of  inquiry  and  investigation  for  teachers  to  learn  not  only  for  themselves  but 
to  apply  to  their  own  classroom  practice.  The  PBL  process  requires  that  teachers  address  questions,  make 
hypotheses,  research  and  investigate  issues,  and  develop  a project  that  addresses  the  problem  or  issue. 

In  conclusion,  the  context  for  which  this  online  PBL  instructional  design  model  was  developed  is  critical  to 
understanding  the  model  itself.  Other  designers  will  perhaps  need  to  adapt  this  model  for  their  own  particular 
contexts,  but  this  will  provide  a starting  place  for  their  efforts. 

Adapting  PBL  for  a Web-Based  Learning  Anytime,  Anywhere  Environment 

Since  the  LTTS  is  a web-based  learning  anytime  anywhere  environment,  using  problem  based  learning  as  a 
design  framework  required  that  we  adapt  PBL  to  work  with  this  environment.  This  resulted  in  the  development  of  a 
new  instructional  model.  In  making  this  adaptation,  we  considered  the  characteristics  of  web-based  learning 
environment,  the  needs  and  characteristics  of  our  learners,  and  their  goal  for  using  the  LTTS. 

First,  several  distance  learning  principles  guided  the  development  of  this  model.  Since  our  learners  are 
from  diverse  backgrounds,  we  cannot  make  common  assumptions  about  them  with  regard  to  prior  knowledge  of 
technology  integration  and  usage,  knowledge  of  terms  or  current  issues,  or  knowledge  of  inquiry  based  learning.  So 
we  designed  a very  structured  PBL  experience  with  built  in  scaffolding  activities  that  help  support  the  learner  in 
solving  the  problem. 

Second,  finding  high  quality  resources  on  the  Web  is  a challenge  for  novices  who  do  not  know  the  research 
or  current  issues.  Therefore,  we  provide  some  high  quality  resources  to  assist  the  learner  in  solving  the  problem. 
These  resources  are  provided  within  activities  as  well  as  separately  in  a resources  section.  Also,  learners  are 
encouraged  to  find  other  high  quality  resources  to  use  in  solving  the  problem  and  contribute  those  to  the  system  for 
others  to  use. 

Third,  the  navigation  of  the  Web  can  inhibit  learning  is  it  is  too  complex  or  difficult  to  use.  If  learners  have 
to  focus  efforts  on  finding  information  or  figuring  out  where  to  go  next  rather  than  learning,  they  can  become  quite 
frustrated.  So  we  designed  a navigational  system  that  illuminates  our  PBL  instructional  model  and  process.  In  fact, 
in  a usability  test  conducted  with  six  teachers  (Kirkley  et.  al.,  2000),  we  found  that  teachers  understood  the  PBL 
flowchart  style  navigation  and  liked  its  consistency. 

With  regard  to  adapting  PBL  for  a learning  anytime  anywhere  environment,  several  adaptations  were  made. 
First,  a traditional  PBL  model  typically  includes  collaborative  group  work  and  tutor  and  a facilitator  who  models 
higher  order  thinking  and  challenges  the  thinking  of  learners.  In  designing  PBL  for  a Web-based  format  where 
learners  are  separated  from  each  other  and  from  the  facilitator  by  time  and  space,  there  is  a challenge  for  the 
instructional  designers  who  want  to  apply  PBL  principles  in  developing  online  learning  environment.  This 
realization  forced  our  design  team  to  consider  the  ways  in  which  learners  and  facilitators  would  want  and  need  to 
communicate  with  each  other. 

In  order  to  implement  PBL  on  the  Web,  we  explored  the  role  that  collaboration  plays  in  the  overall 
experience  of  PBL  but  within  the  framework  of  a learning  anytime,  anywhere  environment.  Within  a face-to-face 
PBL  framework,  learners  have  active,  group-based  roles  at  some  stage  of  the  process  for  the  purpose  of  determining 
solutions  and  synthesizing  knowledge.  In  outlining  PBL,  Boud  (1985)  and  Bridges  and  Hallinger  (1992)  emphasize 
the  importance  of  a group  role  when  learning  stems  from  collaborative  analysis  of  the  problem  and  is  largely 
learner-directed.  Yet  within  a learning  anytime,  anywhere  environment,  it  is  difficult  to  set  up  a collaborative  group 
experience.  With  open  entry/open  exit  structure,  learners  are  completing  modules  at  their  own  pace.  While  learners 
can  participate  and  discuss  issues  with  other  learners,  collaboration  would  be  extremely  difficult. 

The  role  of  the  facilitator  in  LTTS  is  flexible  since  the  type  of  facilitator  and  location  of  the  facilitator  will 
depend  on  the  learner’s  goals  for  completing  a module.  For  example,  if  a learner  is  completing  a module  for 
graduate  credit,  he  may  work  with  a facilitator  at  that  university  in  which  credit  is  being  obtained.  If  a learner  is 
completing  a module  to  receive  continuing  education  credits,  he  may  work  with  his  state  monitor  who  acts  as 
facilitator.  If  a learner  is  completing  a module  to  improve  technology  integration  skills,  he  may  work  with  his 
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district  staff  developer.  With  this  design,  facilitators  will  have  difference  backgrounds,  goals,  and  expectations,  so 
instruction  and  assessment  must  be  well  designed. 

Studies  of  the  cognitive  and  metacognitive  processes  of  students  during  the  initial  problem  analysis  phase 
of  PBL  support  the  view  that  the  role  of  group  interactions  in  PBL  is  to  facilitate  activation  and  elaboration  of 
students’  existing  knowledge  and  so  encourage  conceptual  change  through  cognitive  dissonance  (De  Grave, 
Boshuizen,  and  Schmidt,  1996).  If  this  is  the  function  of  group  interaction  in  PBL,  then,  provided  that  an  alternative 
mechanism  with  an  equivalent  effect  is  introduced,  it  should  be  possible  to  design  effective  PBL  for  individual  use. 

Exploring  the  value  of  PBL  experiences  for  individual  rather  than  collaborative  use  may  be  justified  by  the 
fact  that  professional  practice  is  situated  in  a variety  of  contexts  including  individual  study  as  well  as  collaborative 
and  competitive  teams.  Successful  professional  practice  frequently  depends  upon  individual’s  capacity  to  solve 
problems.  Logically,  educational  experiences  which  develop  that  capacity  should  be  valued.  Individual  PBL 
experiences  may  help  to  address  the  increasing  interest  in  distance  and  flexible  access  to  professional  education  and 
the  increasingly  successful  technology  integration  (Albion  and  Gibson,  1998). 

Thus,  not  diminishing  the  value  of  collaboration  and  facilitation,  we  have  developed  instructional  design 
model  for  individual  web-based  PBL,  using  appropriate  alternatives  which  will  assist  learners  through  the  problem 
solving  process.  Gibson  and  Gibson  (1995)  describe  an  alternative  approach  in  which  a learner  is  engaged  with  a 
problem  individually  and  prepares  a written  analysis  of  the  problem  in  preparation  for  group  interaction.  Within 
LTTS,  this  is  done  through  module  navigation  and  visual  format,  the  use  of  scaffolding  approaches,  such  as 
breaking  the  larger  problem  into  sub-problems,  the  inclusion  of  the  heuristic  aids,  the  integration  of  metacognitive 
self-assessment  tools  as  well  as  various  mechanisms  for  supporting  cooperative  work  on  the  problem  at  a distance. 
All  these  elements  are  intended  to  assist  the  learner  in  the  individual  PBL  web-environment. 

Web-Based  PBL  Instructional  Model 

According  to  our  PBL  model,  which  is  very  similar  to  traditional  PBL  models  (Barrows  and  Myers,  1993), 
the  LTTS  learner  goes  through  the  series  of  phases  in  order  to  finally  generate  a problem  solution.  The  problem 
scenario  begins  with  identification  of  key  concepts  from  the  content  domain  and  a typical  context  in  which  the 
concepts  might  be  used.  This  Presentation  Phase  is  intended  to  situate  the  learner  in  the  problem  context  and  to 
begin  the  process  of  activating  relevant  prior  knowledge.  Additionally,  it  is  in  this  phase  that  learners  have  the 
option  of  beginning  to  customize  their  interpretation  of  the  problem  to  make  the  context  as  specific  as  possible. 

The  Exploration  Phase  provides  opportunity  for  recall  and  reconfiguration  of  prior  knowledge  relevant  to 
the  specific  problem  and  exploration  of  additional,  content  specific  knowledge  and  ‘experience’  gained  during 
problem  solution.  Learners  have  access  to  a collection  of  resources  relevant  to  the  concepts  encapsulated  in  the 
problem.  We  want  them  to  identify  possible  solutions  and  resources  needed  for  understanding  the  problem. 

The  Integration  Phase  emphasizes  relevant  knowledge  transfer,  analysis,  integration,  synthesis  and 
evaluation  of  selected,  content  specific  knowledge  and  problem  based  ‘experience’.  The  problem  in  each  module  is 
divided  into  a series  of  tasks/sub-problems  to  facilitate  scaffolding  by  considering  the  types  of  artifacts,  typically 
documents  of  various  kinds,  which  might  be  produced  by  the  learner  in  association  with  a stepwise  solution  to  the 
problem  situation.  Because  PBL  is  intended  to  increase  the  capacity  of  learners  to  solve  real  problems  and  because 
identifying  critical  elements  may  be  counter-productive,  the  learners  are  required  judgment  in  selection  from  what  is 
provided  and  initiative  in  employing  material  from  alternative  sources  in  order  to  be  able  to  solve  the  problem.  But 
we  need  to  note  that  resource  collection  and  analysis  to  some  extent  is  embedded  into  each  phase  of  the  process. 

The  Solution  Phase  encourages  learners  to  further  integrate  knowledge,  ‘experience’  and  artefacts  gathered 
through  the  problem  solving  process  into  their  cognitive  structures  as  though  products  of  real  experience. 

During  the  Reflection  Phase  learners  are  encouraged  to  conduct  self-assessment  of  their  artifacts  assessing 
the  content  and  organization  of  the  learning  modules  according  to  the  particular  domain  of  technology  integration. 

Each  learning  module  in  the  LTTS  system  is  a self-contained,  problem-based  learning  package,  which 
presents  a scenario,  which  includes  a problem,  resources,  activities,  solution,  and  assessment.  The  problem  may  ask 
the  student  to  develop,  design,  or  critique  something  such  as  an  effective  use  of  technology  to  meet  a need,  the 
resolution  of  an  ethical  dilemma,  or  the  critique  of  a web  interface.  Students  are  asked  to  produce  a product  such  as  a 
report,  or  the  development  of  an  instructional  plan  or  materials  to  address  the  problem.  In  all  cases,  the  problem  is 
flexible  and  enables  learners  to  approach  solving  it  from  their  own  perspective  and  context.  For  example,  rather 
than  being  given  artificial  scenarios,  the  learner  solves  the  problem  using  the  context  of  his  or  her  own  classroom. 
Each  problem  is  authentic  because  it  is  related  to  a significant  issue  in  the  classroom  today. 

The  module  navigation  scheme  visually  represents  the  problem  solving  process,  and  helps  the  learner 
organize  working  process. 
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The  Problem  section  is  intended  to  provide  a situation  description,  a concrete  and  authentic  context  out  of 
which  arises  a problem  to  be  resolved  or  a project  to  be  completed.  It  contains  background  information  that  helps 
establish  the  importance  and  broad  relevance  of  the  problem  to  the  teaching  and  learning  environment.  The  problem 
presentation  contains  enough  information  to  make  the  problem  intriguing,  yet  general  enough  to  allow  the  learner 
freedom  in  determining  the  shape  of  the  solution  or  product.  The  problem  presentation  includes  a specific 
description  of  the  role  learner  will  adopt  while  working  on  the  module.  The  problem  needs  to  be  important  and 
containing  motivating  factors  such  as  mystery,  puzzlement,  novelty,  originality,  and  high  relevance. 

The  purpose  of  the  Resources  section  is  to  provide  the  learner  with  core  materials  that  are  of  high  quality 
and  relevant  to  the  problem.  Our  goal  is  for  the  learner  to  understand  the  resources  as  reflected  in  the  learner’s 
ability  to  apply  them  to  the  problem,  which  contrasts  to  the  traditional  learning  environment  that  measures 
understanding  by  performance  on  a test  rather  than  in  the  ability  to  apply  concepts  and  principles.  Module  resources 
are  primarily  links  to  authentic  materials  (online  research  publications,  projects  web  sites,  online  interviews,  etc.) 
serving  as  a support  to  problem  solving  process.  Resources  may  also  include  references  to  off-line  materials  such  as 
books,  magazines  and  videos.  While  we  provide  learners  with  a list  of  resources,  we  do  not  discourage  them  to  use 
other  materials  they  see  as  relevant  and  useful  in  the  process  of  solving  the  problem. 

Activities  in  problem-based  LTTS  module  serve  as  scaffolded  instruction  that  prepare  the  learner  to  develop 
problem-solving  strategies.  Due  to  the  challenges  of  online  environment,  and  considering  the  lack  of  instructor  or 
facilitator,  we  divide  problem  into  sub-problems  or  tasks  that  scaffold,  model  and  coach  the  learner  in  his/her 
thinking  and  learning.  Each  activity  is  built  around  the  issue  relevant  to  the  overall  problem  and  includes  links  to 
resources  presenting  this  issue.  The  learner  is  expected  to  complete  a written  analysis  of  some  aspects  of  the 
problem  for  the  purposes  of  synthesizing  knowledge  and  determining  solutions.  All  submitted  works  are  stored  in 
the  system  database  and  the  learner  has  an  access  to  them  at  the  later  stages  while  preparing  the  final  product.  The 
learner  has  an  opportunity  to  modify  and  resubmit  his/her  work  and  retrieve  his/her  works  in  the  Solution  section  in 
order  to  support  the  final  product. 

When  the  learner  has  completed  the  tasks  specified  in  module  Activities  he/she  is  expected  to  prepare  a 
final  product,  Solution  for  the  problem.  The  solution  can  be  prepared  in  the  format  of  a paper,  project,  job-aid  or 
instructional  materials  addressing  the  problem.  The  final  product  must  address  the  major  decisions  that  were  made 
and  should  be  based  on  or  cite  research,  theory,  or  practice  that  grounds  the  decision  making  in  the  literature.  In 
Solution  section  the  learner  has  an  opportunity  to  review  his/her  earlier  work  from  Activities  and  create  a final 
product. 

The  purpose  of  the  Assessment  section  is  to  encourage  and  help  structure  the  learner’s  reflection  on  his/her 
performance  through  a series  of  reflection  questions.  We  recognize  the  importance  of  reflective  questions  for 
understanding  the  learners’  thinking  and  assisting  them  move  through  the  experience.  In  our  design,  we  adopted  the 
portion  of  Naidu  and  Oliver’s  PBL  model  (1996)  that  deals  with  reflection  at  multiple  points  in  the  process.  We 
included  two  types  of  reflective  questions  that  proved  to  be  most  important:  check-up  questions,  and  stepping  back 
questions  (Hmelo  and  Ferrari,  1997).  Check-up  questions  are  the  ones  that  help  students  think  about  they  goals  as 
they  work.  In  LTTS  module  these  questions  take  the  form  of  self-checks  integrated  into  each  activity.  The  second 
set  of  questions,  stepping  back,  take  the  form  of  final  reflection  questions  that  ask  the  learners  to  reflect  on  the 
whole  process  of  solving  the  problem.  Reflection  is  considered  a critical  tool  for  synthesis  as  well  as  for  facilitating 
students’  forward  motion  in  the  problem  environment  (Orrill,  2000). 

Reflection  on  the  Instructional  Model 

Admittedly,  the  structure  we  used  for  web-based  problem  solving  process  is  much  more  rigid  than  traditional  face- 
to-face  PBL  environment.  However,  Abrami  and  Bures  (1996)  recognize  that  setting  the  agenda  for  students 
learning  at  a distance  may  be  problematic  since  learners  structure  to  help  in  planning  and  managing  projects.  This  is 
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why  we  structured  the  problem  solving  process  by  dividing  the  problem  into  a set  of  sub  problems,  or  activities. 
Wegner,  Holloway,  and  Grader  (1997)  also  confirm  the  need  for  structured  support  in  their  online  PBL  experience. 
Their  issues  focus  on  the  development  of  particular  content  knowledge.  They  use  some  imposed  structures  such  as 
process-oriented  questions  and  lists  of  key  terms  and  concepts  in  order  to  help  their  students  move  through  the  PBL 
experience  and  reach  the  desired  outcomes.  While  this  may  have  prevented  the  students  from  arriving  at  the 
“knowledge  abstraction’  phase  of  PBL  (Barrows,  1985),  it  likely  helps  them  stay  on  track  to  finishing  the  problem. 

Compromising  certain  aspects  of  the  problem-solving  experience,  we  are  also  scaffolding  learners  to  help 
them  work  in  web-based  environment.  However,  while  providing  more  rigid  structuring  strategies  we  do  not 
eliminate  the  most  of  the  critical  elements  of  PBL.  A community  of  peers  and  a facilitator  provide  ongoing  support 
to  learners  working  on  the  problems  individually.  Corrent-Agostinho,  Hedberg,  and  Lefoe  (1998)  emphasize  that 
the  most  successful  learning  experiences  are  those  in  which  the  environment  both  structured  and  well  facilitated. 

The  role  of  the  facilitator  is  not  to  inform  but  to  model  higher  order  thinking  and  to  challenge  the  thinking 
of  learners  (Boud,  1985;  Savery  and  Duffy,  1995).  Studies  of  student  interaction  with  peers  in  the  PBL  environment 
(De  Grave,  Boshuizen,  and  Schmidt,  1996)  suggest  that  exposure  to  different  ideas  in  the  group  leads  to  conceptual 
change.  The  group  interactions  serve  to  encourage  activation  and  elaboration  of  existing  knowledge  and  integration 
of  alternative  views.  Providing  the  style  of  support  which  the  learner  typically  receives  from  the  facilitator  and  peers 
in  the  face-to-face  PBL  environment  presents  an  obvious  challenge  to  the  web-based  PBL  design.  In  our  design,  we 
are  considering  several  options  to  satisfy  the  learners’  need  for  group  interaction  and  facilitation  of  their  activities. 
The  LTTS  design  team  is  developing  various  mechanisms  (discussion  forums,  chat  rooms)  to  support  the  process  of 
building  a community  of  learners  interested  in  the  similar  problematic  issues,  discussing  authentic  experience, 
creating  mentor-type  relationships  and  teams  to  work  together  on  the  same  problem.  Another  strategy,  which  would 
compliment  the  group  interactions,  is  incorporating  a feedback  mechanism  through  which  after  completing  a task 
the  student  will  obtain  access  to  a collection  of  responses  containing  varying  interpretations  of  a problem.  It  is 
anticipated  that  exposure  to  a collection  of  responses  in  this  way  will  have  effects  similar  to  interactions  among  a 
group  of  learners  with  differing  vision  of  a problem  and  its  solution.  We  hope  to  develop  such  an  environment 
where  the  learner  utilizing  the  individualized  PBL  model  in  a web-based  learning  environment  will  be  able  to 
receive  both  human  and  technology  support. 

At  each  stage  of  the  design  process,  we  used  rapid  prototyping  techniques  to  ensure  that  the  overall 
scenario  was  plausible  and  the  problem  process  flowed  naturally  according  to  learners’  perspective  (Tripp  and 
Bichelmeyer,  1990).  This  was  done  through  having  inservice  teachers  examine  the  modules  in  depth  and  make 
suggestions.  Also,  usability  testing  was  conducted  with  six  teachers,  who  are  the  real  audience  for  LTTS. 

Conclusion 

This  is  a new  instructional  model  for  Web-based  PBL.  Obviously,  it  is  not  the  same  as  traditional  face-to-face 
PBL,  and  it  is  not  our  purpose  to  duplicate  the  traditional  face  to  face  PBL,  which  has  the  strengths  of  group 
collaboration.  Our  goal  is  to  take  the  best  of  PBL  learning  and  develop  a new  instructional  model  that  would  work 
within  a Web-based  learning  anytime,  anywhere  environment.  In  doing  this,  we  have  to  consider  both  the  strengths 
and  constraints  of  web-based  instruction,  learner  characteristics,  and  the  purpose  of  the  LTTS,  which  is  professional 
development.  We  need  to  understand  the  strengths  and  limitations  of  the  online  environment  and  learn  how  to 
operate  within  these.  We  also  have  to  understand  what  aspects  of  PBL  will  work  in  these  environments  and  will 
enhance  professional  development  as  well  as  how  PBL  needs  to  be  adapted  to  fit  the  learning  environment  in  which 
it  occurs. 

This  instructional  model  is  just  one  of  many  possible  implementations  of  PBL  on  the  Web.  As 
communication  technologies  advance,  there  will  be  future  instructional  design  models  of  PBL  that  are  even  more 
innovative.  McLoughlin  and  Oliver  (1999)  argue  that  we  need  to  develop  online  tools  to  support  parallel  problem 
solving,  simulating  course  material,  information  exchange,  database  creation,  and  case-based  projects.  As  new  tools 
are  created,  instructional  designers  will  need  to  develop  enhanced  instructional  models  that  facilitate  and  support  the 
inquiry  and  problem  solving  processes  within  the  context  of  the  areas  being  studied. 
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Abstract 

A review  of  research  in  distance  education  indicates  thatresults  are  consistent  in  that  there  is  no  significant 
difference  in  achievement  contributable  to  the  delivery  system . And  it  is  this  * no  significant  difference  ’ that  seems  to 
be  quoted  most  frequently.  Thisperspective  represents  a very  narrow  vision  of  what  research  is  actually  occuring  in 
the  field  of  distance  education.  This  paper  attempts  to  look  beyond  or  behind  this  no  significant  difference  in 
achievement  by  examining  research  related  to  threeother  factors:  learner  interatction  and  control,  two  different 
approaches  to  multiple  learning  styles  and  utilizing  intelligent  agents  to  facilitate  interaction  and  collaboration. 
The  approach  has  bee  taken  to  not  simply  reviw  what  has  been  studied  but  to  examine  areas  for  future  study  that 
will  make  significant  contributions  to  the  field. 

Introduction 

Research  in  distance  education  has  tended  to  follow  the  same  trends  as  research  with  other  new 
technologies.  The  first  factor  that  is  explored  is  related  to  learner  outcomes.  The  temptation  is  there  to  try  to  prove 
that  the  use  of  new  delivery  systems  result  in  higher  student  achievement.  The  multitude  of  media  comparison 
studies  conducted  in  the  area  of  distance  education  have  shown  repeatedly  that  distance  education  is  at  least  as 
effective  as  traditional  education  in  regards  to  learner  outcomes  ( Hanson,  et.  al,  1996,  Russell,  1999,  Saba,  2000). 
In  fact,  several  studies  have  produced  results  that  indicated  distance  education  students  had  a higher  achievement 
than  those  in  traditional  settings.  However,  a review  of  the  literature  supports  a perspective  of  research  in  distance 
education  that  is  examining  this  phenomenon  from  multiple  approachers.  This  paper  presents  an  examination  of  the 
research  in  distance  education  examining  factors  beyond  achievement  including  emerging  technologies,  learning 
styles  and  interaction. 

The  first  section  will  look  at  using  CAI  to  support  K-12  distance  education.  While  CAI  appears  to  be 
decreasing,  it  offers  intriguing  possibilities  when  explored  for  its  distance  education  potential  particularly  in  the  K- 
12  setting.  Learning  styles  appear  frequently  in  the  distance  education  literature.  However,  the  utilization  of  multiple 
models  and  instruments  makes  interpreting  and  generalizing  the  results  difficult.  The  second  section  explores  a 
classification  system  for  learning  styles  that  would  prove  useful  to  help  interpret  the  research  related  to  learning 
styles  in  distance  education.  The  third  section  explores  learning  styles  in  one  specific  setting.  The  use  of  Web-based 
instruction  continues  to  increase  but  little  has  been  studied  in  regards  to  learning  styles  and  designing  web-based 
instruction  that  supports  multiple  learning  styles.  The  fourth  and  final  section,  examines  the  emerging  technology  of 
intelligent  agents  and  what  impact  this  might  have  on  changing  the  paradigm  of  distance  education. 

Using  CAI  to  Facilitate  Distance  Education  in  K-12 

Since  1985,  distance  education  has  been  growing  as  a means  of  teaching  and  learning.  Distance  education 
enables  bringing  knowledge  to  students  regardless  of  the  limitation  of  time  and  geographical  location.  It  has  been 
typically  defined  both  as  a synchronous  mode  and  as  an  asynchronous  mode.  Contemporaneously,  Computer- 
Assisted  Instruction  (CAI)  has  been  evolved  since  1960s.  CAI  is  an  effective  intervention  for  improving  students’ 
achievement  in  different  subject  areas  (Christmann,  Badgett,  & Lucking,  1997).  Researchers  have  dedicated  many 
efforts  and  money  to  study  and  improve  the  instructional  design  of  CAI.  However,  with  the  rapid  development  of 
distance  education,  the  use  of  CAI  seems  to  be  decreasing  in  popularity.  It  is  no  longer  the  spotlight  compared  to 
the  foci  on  other  delivery  methods  within  distance  education.  Nevertheless,  CAI  has  another  chance  to  gain  the 
attention  of  instructors  and  instructional  designers  again  with  the  increasing  use  of  distance  education  within  the 
context  of  K-12. 
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Instructional  Forms 

Lauzon  & Moore  (1989)  thought  that  a successful  application  is  dependent  upon  understanding  how  the 
technologies  can  be  used  to  enhance  distance  learning.  Hence,  it  is  necessary  to  review  instructional  forms  prior  to 
adopting  a specific  instructional  design  for  distance  education.  Kowitz  and  Smith  cited  in  Lauzon  &Moore  (1989) 
mentioned  the  three  forms  of  instruction  represent  positions  on  two  dimensions:  the  density  of  content  to  be  learned 
and  the  styles  of  human  interaction.  Different  instructional  forms  require  different  styles  of  leamer/instructor 
interaction. 

The  first  form  represents  learning  the  basics.  It  is  characterized  by  instructor  control  of  the  development, 
design,  and  evaluation.  The  second  form  assumes  students  an  active  role  in  their  education.  The  instructor  acts  as 
an  expert  and  becomes  more  consultative  with  students.  It  is  characterized  by  instructor  control  as  well  as  learner 
control  of  field  study.  The  third  instructional  form  is  characterized  by  experts  seeking  to  improve  and  master  their 
existing  knowledge  and  performance.  It  is  more  toward  learner  control. 

Factors  Influencing  Learning,  Interactions,  and  Control  in  Distance  Education 

Vrasidas  and  Mclsaac  (1999)  concluded  the  structure  of  the  online  transaction,  class  size,  feedback 
provided  to  students,  and  students’  prior  experiences  were  the  major  factors  to  influence  interactions.  Baynton 
(1992)  indicated  that  the  factors  of  “control”  in  distance  education  are  competence,  independence,  support,  time 
flexibility,  value  orientation,  and  access  to  resources. 

By  comparing  aspects  of  studies,  we  can  make  an  assumption  that  we  are  comparing  students’  expectations 
(Vrasidas  & Mclsaac,  199;  Baynton,  1992)  and  teaching  behaviors  (Mckenzine  et  al.,  1998).  Students’  expectations 
are  more  in  terms  of  self  preferences,  while  teaching  behaviors  are  more  in  terms  of  management  and  instructional 
concerns.  Nevertheless,  requiring  and  providing  support  and  feedback  are  congruent. 

Distance  Education  in  K-I2 

Currently,  most  of  the  attention  given  distance  learning  programs  in  K-12  schools  is  focused  on 
synchronous  modes  of  instruction  (Barker  & Dickson,  1996).  Schools  are  using  distance  education  technologies  to 
help  them  offer  both  elective  and  required  courses  for  which  a certified  teacher  is  not  available  or  in  situations  where 
low  student  enrollments  do  not  qualify  a full-time  teacher  being  hired.  Barker  and  Dickson  (1996)  indicated  that  the 
most  common  technologies  used  in  distance  education  programs  in  K-12  have  been  satellite-delivered  instruction, 
cable  television,  and  computer  audiographics.  Even  so,  Barker  and  Dickson  also  indicated  that  the  use  of  the 
Internet  and  the  Web  to  assist  distance  education  has  been  emerging. 

Characteristics  of  CAI 

It  is  essential  that  CAI  applications  be  examined  in  order  to  seek  consistency  with  instructional  forms  as 
well  as  interaction  styles  and  controls  mentioned  above.  Thesaurus  of  ERIC  descriptors  cited  in  Fourie  (1999) 
defines  that  CAI  stands  for  an  interactive  instructional  technique  in  which  a computer  is  used  to  present  instructional 
material.  CAI  can  monitor  learning  as  well  as  present  corresponding  instructional  material  to  what  individual 
learner  needs.  Lockard,  Abrams,  and  Many  (1997)  described  the  nature  of  CAI  as  the  following: 

Computer-assisted  instruction  (CAI)  is  the  most  common  term  for  the  interaction  of  a learner  with 
a computer  in  a direct  instructional  role.  CAI  software  provides  instruction  in  some  particular 
content  in  any  of  a variety  of  formats,  with  or  even  without  any  involvement  of  a human  teacher. 

(p.  190) 

The  major  characteristics  of  CAI  are  interaction,  flexibility,  and  meeting  student  needs  (Lockard,  Abrams,  & Many, 
1997).  CAI  engages  learners  interacting  directly  and  continually  with  computers.  Learners  take  active  roles  instead 
of  passive  roles  in  their  learning  processes.  It  provides  feedback  to  learners  as  well.  Its  flexibility  allows  teachers 
involving  in  teaching  both  higher-order  problem-solving  and  simple  cognitive  learning.  Furthermore,  it  can 
response  to  different  needs  due  to  different  levels  of  competencies  that  learners  hold.  The  advantages  and 
disadvantages  of  CAI  in  terms  of  distance  education  are  discussed  in  the  following. 

Advantages.  The  advantages  of  CAI  are  (Fourie,  1999;  Lockard,  Abrams,  & Many,  1997;  Luzon  & Moore, 
1989;  Daniel,  1999): 

1 . Interaction:  CAI  engages  learners  interacting  directly  and  continually  with  computers. 

2.  Immediate  feedback:  CAI  is  capable  of  providing  immediate  feedback  to  learners. 

3.  Self-pacing:  Learners  can  learn  at  their  own  paces  with  CAI. 

4.  Visualized  effects  and  sound:  Integrated  effects  can  impress  learners  to  keep  their  retention  and  extend 
the  duration  of  their  learning. 

5.  Branching  for  different  interests  or  difficulties:  Learners  can  go  through  the  order  depending  on  their 
own  learning  styles  and  entry  levels. 
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6.  Providing  sufficient  drill,  practice,  simulations,  and  games:  CAI  provides  plenty  of  opportunities  to 
learners  to  do  drill,  practice  skills,  simulate  problems,  and  have  fun  while  learning. 

7.  Independent  tutorial:  Learners  can  go  through  the  tutorial  when  they  need. 

8.  Allowing  a large  number  of  learners:  CAI  applications  can  be  duplicated  and  distributed  easily. 

9.  Time  flexibility:  Learners  can  use  CAI  at  their  own  convenience. 

Disadvantages.  The  disadvantages  of  CAI  includes  ((Fourie,  1999;  Lockard,  Abrams,  & Many,  1997; 
Luzon  & Moore,  1989;  Joseph,  1999): 

1.  Cost  to  develop:  It  is  expensive  to  develop  a new  CAI  application  and  takes  much  time  and  many 
efforts  of  people  involved. 

2.  Computer  platforms:  If  CAI  only  used  for  distance  education  by  distributed  learning,  different 
computer  platforms  that  learners  have  should  be  considered  and  can  be  a problem. 

3.  No  standards  to  apply  applications:  There  is  no  standard  for  what  CAI  application  to  apply  for  courses. 

4.  Requiring  computer  skills:  Certain  computer  skills  are  needed  while  working  within  an  CAI 
application. 

5.  Lack  of  synchronic  oral  communication. 

6.  Eyestrain:  It  is  easy  to  get  eyestrain  after  a long  period  of  watching  the  computer  screens. 

However,  Olcott  (1997)  argued  that  distance  education  does  not  necessarily  save  money  in  the  short  term  if 
we  view  distance  education  from  a value-added  perspective. 

Discussion  and  Conclusion 

Olcott  (1997)  stated  the  shift  to  competency-based  assessment  models  that  certify  learning  through  mastery 
of  specified  skill  competencies  rather  than  learning  based  on  credit  hours  and  seat  time  has  impacted  K-12  school 
restructuring  initiatives.  The  shift  may  use  CAI  applications  to  achieve  its  goals  since  CAI  applications  can  provide 
students  with  what  individual  needs  and  enhance  learning.  CAI  can  work  for  K-12  students  in  the  diverse  subject 
areas  and  further  in  the  distance  education  context. 

CAI  is  well  known  for  its  ability  to  address  the  independence  of  learners.  This  aspect  relates  well  to  how 
distance  education  works  for  most  participants.  Through  distance  education,  K-12  schools  can  obtain  collaborative 
and  cooperative  teaching  and  learning.  CAI  has  the  potential  to  enrich  and  facilitate  distance  education  for  K-12 
schools.  With  the  help  of  CAI,  the  use  of  distance  education  in  K-12  could  be  extended  to  every  single  possible 
geographical  and  content  area. 

CAI  is  congruent  with  how  distance  education  works.  Traditional  distance  education  programs  of  the  past 
10  years  operate  mostly  in  the  synchronous  mode;  however,  increasingly,  programs  are  being  offered  in  the 
asynchronous  mode  especially  because  of  the  rapid  development  of  the  Internet  and  the  World  Wide  Web  (Baker  & 
Dickson,  1996).  Garrison's  opinion  (1986)  cited  in  Lauzon  and  Moore’s  observation  (1989)  indicated  that  the  new 
generation  of  delivery  system  for  distance  education  should  be  capable  of  both  asynchronous  group  and 
individualized  instruction  and  integrate  the  communications  network  with  computer-based  instruction.  In  other 
words,  CAI  combining  with  the  Internet  and  the  World  Wide  Web  is  the  ideal  application  for  distance  education.  It 
is  evidenced  that  Web  browsers  are  powerful  programs  which  can  integrate  various  types  of  media.  Web  browsers 
provide  an  inexpensive  and  widely  available  application  that  can  combine  text,  graphics,  audio,  video,  data,  and 
programming  within  the  same  software  program  (Daneil,  1999).  Consequently,  CAI  on  the  Web  does  not  need  to 
address  the  problem  caused  by  the  different  platforms  of  computers 

It  is  easy  to  update  or  upgrade  CAI  applications  via  the  Internet  so  as  to  save  cost  and  time  from  publishing 
and  distribution.  While  learning  with  CAI,  learners  will  have  easy  access  and  jump  to  different  resources  due  to  the 
connections  with  the  Internet  and  the  context  of  the  World  Wide  Web.  Learners  also  can  acquire  immediate  use  of 
e-mail,  chat-room,  and  even  Internet  phone  to  have  online  communications  with  others.  Developing  CAI  for  the 
Web  also  facilitate  collaboration  by  geographically  dispersed  institutions  (Daneil,  1999).  This  will  increase 
collaboration  and  cooperation  between  schools. 

In  sum,  CAI  applications  for  distance  education  for  K-12  students  should  integrate  with  the  Internet  and  the 
Web.  The  Internet  provides  synchronous  settings  as  well  as  asynchronous.  The  implements  and  potentials  of  CAI 
on  the  Web  in  distance  education  for  K-12  need  to  be  studied  by  researchers,  instructors,  and  designers  much  farther 
in  the  future. 


The  Influence  of  Learning  Styles  in  Distance  Education 

Research  studies  in  distance  education  have  measured  learning  styles  using  different  instruments  and  have 
classified  these  styles  according  to  various  schema.  Most  of  these  studies  used  subjects  that  were  either  not 
representative  of  the  population  or  were  samplings  of  convenience.  These  conditions  complicate  the  comparisons  of 
the  resulting  data.  In  order  to  compare  learning  styles  across  these  research  studies,  the  learning  style  instruments 
and  classifications  need  to  be  related  to  general  models.  Claxton  & Murrell  (1987)  have  classified  learning  style 
instruments  into  three  broad  learning  style  models:  1)  instructional  preference,  2)  social  interaction,  and  3) 
information  processing. 

Instructional  preference  models  address  motivation  and  persistence  of  the  learner.  Social  interaction  models 
are  learning  style  models  that  address  the  learner’s  need  for  an  interchange  of  ideas  or  knowledge  with  one  or  more 
people  concerning  the  learning  material.  Information  processing  model  address  the  influence  of  the  structure  of  the 
instruction,  an  individual’s  self-concept  as  a learner,  distinct  personal  goals,  and  expectations  of  success  or  failure 
on  the  amount  of  information  learned.  Learning  styles  are  the  combination  of  the  developmental,  cognitive,  and 
affective  factors  that  influence  the  way  in  which  we  process  and  perceive  information. 

Instructional  Preference  Models 

Instructional  preference  models  are  learning  style  models  that  are  associated  with  environmental  and 
emotional  preferences.  Emotional  preferences  that  address  motivation,  persistence,  and  structure  have  been 
extensively  studied  in  distance  education.  Research  has  shown  that  knowledge  acquired  through  activities  that 
motivate  the  individual  is  learned  more  deeply  than  rote  memorization  (Cognition  and  Technology  Group  at 
Vanderbilt,  1993).  Persistence,  which  is  a function  of  motivation,  is  the  extent  to  which  an  individual  continues  to 
do  an  activity. 

In  distance  education,  many  research  studies  have  investigated  the  persistence  of  learners  either  within  a 
course  or  within  a program.  These  studies  have  examined  previous  educational  level,  age,  gender  (Coggins,  1988; 
Powell,  Conway,  & Ross,  1994;  and  Richards  & Ridley,  1997),  employment,  illness,  family  problems  (Coggins, 
1988;  Gibson,  1996;  and  Powell  et.al,  1994),  support  programs,  tutors,  and  quality  of  instructional  material  (Powell 
et.  al.,  1994).  Many  different  schemes  have  been  used  to  provide  an  explanation  or  prediction  of  attrition,  but  the 
single  point  of  agreement  is  that  attrition  is  multivariate  in  nature  (Gibson  & Graff,  1992).  When  examining  how 
learning  styles  affect  persistence,  we  must  look  at  emotional  elements  that  relate  to  motivation.  Motivation,  whether 
intrinsic  or  extrinsic,  helps  to  drive  the  student  to  learn  a particular  piece  of  information.  With  intrinsic  motivation, 
a learner  is  driven  internally  by  interest  and  curiosity  and  learning  that  piece  of  information  is  important  to  the 
learner  himself.  A learner  that  is  extrinsically  motivated  is  driven  to  learn  a particular  piece  of  information  by  a 
requirement  imposed  by  some  outside  force.  The  importance  of  that  requirement  to  the  learner  will  determine  that 
amount  of  drive  to  learn  the  information. 

Another  factor  that  is  common  in  a number  of  distance  education  persistence  studies  pertains  to  the 
learner’s  perception  of  his  ability  to  succeed  in  the  educational  setting  (Gibson,  1996;  and  Powell  et.  al.,  1994).  A 
learner’s  academic  self-concept  is  important  as  a predisposing  characteristic  in  the  learner’s  ability  to  complete  a 
distance  education  program  (Coggins,  1988).  The  confidence  a learner  has  in  his  ability  to  succeed  in  a course  could 
affect  the  learner’s  motivation 

Coggins  also  stated  that  major  motivational  factors  influencing  a learner’s  persistence  in  a distance 
education  course  are  goal-clarity  and  course  relevance.  These  motivational  factors  can  be  intrinsic  or  extrinsic 
factors  and  are  significant  influences  on  distance  education  persistence.  The  students  who  intend  to  earn  a degree 
and/or  who  were  taking  a distance  course  that  was  relevant  to  their  degree  program  were  more  likely  to  complete  the 
distance  education  course  or  program  (Coggins,  1988;  and  Powell  et.  al.,  1994). 

Social  Interaction  Models 

Social  interaction  has  been  shown  to  be  a significant  factor  in  determining  distance  education  success. 
Individuals  who  prefer  a collaborative  learning  style,  where  interaction  with  other  learners  is  important,  are  less 
likely  to  succeed  in  a distance  education  environment  (Dial  & Cartnal,  1999). 

A recent  study  examined  the  implementation  of  groupware  into  an  asynchronous  learning  environment  and 
found  that  groupware  facilitated  the  group  learning  process  (Becker  & Dwyer,  1998).  This  study  also  investigated 
visual/verbal  learning  preferences  of  the  students.  A comparison  of  visual/verbal  learning  styles  to 
independent/dependent  learning  styles  has  not,  to  my  knowledge,  been  conducted  with  respect  to  distance  education. 
However,  accepting  the  fact  that  dependent  learners  learn  best  when  interacting  with  other  students  or  teachers,  a 
preference  for  verbal  learning  would  include  dependent  learners.  Interestingly,  Becker  and  Dwyer  found  that 
students  who  were  categorized  as  more  visual  learners  perceived  an  added  benefit  from  utilizing  groupware  and 
considered  the  groupware  to  be  valuable  in  the  group  learning  process. 
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Information  Processing  Models 

The  amount  of  information  processed  and  the  amount  of  information  learned  are  influenced  by  an 
individual’s  self-concept  as  a learner,  distinct  personal  goals,  expectations  of  success  or  failure,  and  preference,  or 
lack  of  preference,  for  structured  learning  activities  (Gibson,  1996;  Wagner  and  McCombs,  1995;  Dunn,  Griggs, 
Olson,  Gorman,  & Beasley,  1995).  Research  studies  in  distance  education  have  linked  field 
dependence/independence  with  academic  achievement  (Ching,  1998;  and  Powell  et.  al.,  1994;).  Learners  who  prefer 
a more  structured  learning  environment  are  learners  who  prefer  a dependent  learning  style  (field  dependent).  These 
learners  are  less  likely  to  succeed  in  a distance  education  environment  (Powell. et.al.,  1994;  ).  As  the  degree  of 
learner  control  of  the  distance  education  environment  increases  (or  structure  is  decreased),  dependent  learners 
reported  that  they  “got  lost”  when  studying  or  were  unsure  of  the  instructor’s  expectations  (Ching,  1998;  and 
Gibson,  1996). 

In  a study  examining  the  amount  of  learner  control  in  a hypermedia  environment,  active  (independent) 
learners  performed  best  with  high  levels  of  learner  control,  and  reflective  (dependent)  learners  performed  best  with 
low  levels  of  learner  control  (Rasmussen  & Davidson-Shivers,  1998).  In  a study  of  forty-eight  sixth-graders,  a 
structured  presentation  of  material  facilitated  learning  better  than  an  unstructured  presentation  of  material  when 
using  computer  based  instruction  (Yang  & Chin,  1996). 

In  a distance  education  nursing  program,  a study  demonstrated  that  a field  dependent  learning  style  is  a 
good  predictor  of  poorer  academic  performance  (Ching,  1998).  This  study  also  demonstrated  that  development  of 
field  independence  could  be  influenced  by  the  curriculum  and  teaching  approach.  Ching  compared  the  field 
independence  of  students  beginning  distance  education  study  to  their  field  independence  at  the  end  of  one  year  of 
study  and  found  that  field  independence  increased  in  these  students.  Ching  attributed  this  increase  to  the  distance 
education  environment  on  the  basis  that  this  distance  education  course  was  taught  to  isolated  students  that  had  no 
formal  means  of  student-to-student  interaction.  However,  many  students  did  indicate  that  they  created  their  own 
student  study  groups  that  would  foster  the  field  dependent  students’  learning. 

Conclusion  and  Future  Studies 

Learning  styles  are  a preference  in  how  a learner  interprets  and  processes  information.  Most  of  the  learning 
style  instruments  measure  learning  styles  in  terms  of  binomial  opposites  (i.e.,  dependent/independent  and 
verbal/visual).  However,  learning  styles  are  not  absolute  opposites;  they  are  measurements  of  a continuum  between 
these  binomial  opposites.  Learners  prefer  a certain  mode  of  learning,  and  are  not  locked  into  one  mode  of  learning. 
In  the  study  of  Chinese  nursing  students,  it  was  demonstrated  that  a student’s  learning  style  could  change  over  time 
(Ching,  1998). 

One  consistent  finding  in  the  research  on  learning  styles  in  distance  education  was  the  outlook  for  the 
dependent  learner.  The  independent  learner  was  more  likely  to  succeed  in  a distance  education  course  or  program 
than  a dependent  learner.  The  dependent  learner  may  be  a good  predictor  of  poorer  academic  performance. 

To  date  in  distance  education  research,  most  of  the  studies  in  learning  styles  have  used  samplings  of  convenience. 

Also  most  of  these  studies  were  conducted  on  students  who  chose  to  take  the  distance  education  course 
over  a traditional  course  offered  at  the  same  institution.  In  general,  independent  students  usually  self-select  to  the 
distance  education  course  while  dependent  student  self-select  to  the  face-to-face  courses.  With  the  use  of  distance 
education  in  universities  to  deliver  courses  throughout  a system  where  students  are  distributed  among  campuses  in 
different  cities,  the  choice  of  taking  a traditional  face-to-face  course  may  not  exist.  Therefore,  future  studies  should 
examine  the  dynamics  of  learning  styles  when  the  student  has  no  choice  but  a distance  education  course  or  program. 

Addressing  Multiple  Learning  Styles  in  Web-based  Instruction 

The  purpose  of  this  literature  review  is  to  report  current  research  addressing  how  Web-based  instruction 
and  the  technological  capabilities  of  the  World  Wide  Web  have  been  used  to  meet  the  individual  learning  styles  of 
students.  This  review  will  look  at  research  in  the  areas  of  learning  styles  and  World  Wide  Web  technology. 

The  area  of  Learning  Styles  has  been  researched  extensively  throughout  the  past  fifty  years.  Much  of  this  research 
has  been  conducted  in  the  traditional  classroom  setting.  The  majority  of  research  supports  matching  the  individual's 
learning  style  with  a complementary  mode  of  instruction.  Researchers  agree  that  learning  styles  represent  the  ways 
in  which  individuals  interpret,  process,  understand,  and  integrate  information.  Learning  styles  are  defined  in  many 
different  categories  and  inventories,  none  of  which  are  standardized  across  the  research.  This  makes  it  difficult  to 
apply  general  conclusions  across  the  research  findings.  Since  much  of  the  research  on  learning  styles  supports 
teaching  students  to  their  individual  learning  styles,  for  greater  academic  achievement,  and  shows  support  for  the 
need  to  address  multiple  learning  styles  in  every  instructional  lesson,  this  literature  review  is  not  limited  to  any 
specific  learning  styles  categories  or  inventories. 
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The  research  in  the  combined  areas  of  learning  styles  and  Web-based  instruction  is  very  limited.  The 
research  that  has  been  reported  used  mainly  undergraduate  college  students  as  samples.  Little  is  known  or  has  been 
reported  about  addressing  learning  styles  in  Web-based  instruction,  for  K-12  students.  This  appears  to  be  an  area 
requiring  further  research. 

Learning  Styles 

Extensive  research  has  been  conducted  in  the  area  of  learning  styles  in  the  traditional  classroom  setting.  A 
great  majority  of  this  research  has  supported  the  fact  that  each  individual  differs  in  the  way  they  interpret,  process, 
understand,  and  integrate  information.  Research  shows  that  people  exhibit  significant  individual  differences  in  the 
cognitive  processing  styles  that  they  adopt  in  problem  solving  and  other  similar  decision  making  processes 
(Robertson,  1985).  Individual  differences  in  student  learning  are  categorized  into  learning  styles.  Learning  styles  are 
the  ways  in  which  the  brain  interprets  processes,  understands,  and  integrates  information.  Dunn,  Dunn,  and  Price 
(1989),  state  that  classrooms  need  to  concentrate  more  upon  individual  learning  styles  because  students  tend  to  learn 
and  remember  better  and  enjoy  learning  more  when  they  are  taught  in  a way  that  takes  into  account  for  their  learning 
style  preferences.  When  developing  instruction  to  accommodate  learning  styles,  the  instructional  developer  must 
know  what  delivery  mode  best  suits  each  learning  style  type.  When  applying  learning  styles  to  the  curriculum,  Dunn 
& Dunn  (1978)  state  that  students  should  be  taught  to  their  dominant  learning  style  and  then  followed  with  their 
second  strongest  learning  style.  Auditory  learners  learn  best  when  they  listen,  read,  and  then  take  notes.  Visual 
learners  learn  best  when  they  read,  look  at  visual  aids,  took  notes,  and  then  listened  to  a lecture  on  the  material. 
Tactile  learners  require  manipulative  materials  in  which  they  can  use  to  construct  and  then  read,  write,  and  listen  to 
the  material.  Kinesthetic  learners  prefer  exposure  to  the  real  world,  such  as  field  trips,  followed  by  reading,  writing, 
and  listening  to  the  material.  A case  study,  by  Hodges  (1982),  shows  how  putting  these  learning  styles  strategies  into 
action  can  increase  student  achievement.  The  study  was  conducted  with  Junior  High  school  students,  from  extremely 
low  socioeconomic  backgrounds,  who  were  not  responding  to  conventional  strategies  for  learning.  Each  student  was 
given  a Learning  Style  Inventory  test  and  an  individual  curriculum  was  matched  with  the  student’s  dominant 
learning  style.  Results  indicate  that  eighty-five  percent  of  the  students  substantially  increased  their  achievement 
levels  in  reading  and  math  compared  with  their  previous  performance  in  a typical  school  setting.  The  data  revealed 
that,  when  learners  are  taught  through  methods  that  complement  their  learning  style  preferences,  learners  become 
more  motivated  and  have  higher  academic  achievement. 

Robotham  (1995)  questions  the  practice  of  placing  students  into  pre-specified  learning  style  categories. 
The  author  believes  that  this  causes  students  to  become  stagnant  in  their  ability  to  understand  how  they  learn.  The 
author  believes  that  forcing  students  to  learn,  using  different  learning  styles,  can  help  them  grow.  Snider  (1990)  also 
states  that  it  is  good  practice  to  recognize  and  accommodate  individual  differences  and  to  present  information  in  a 
variety  of  modalities.  This  research  shows  support  for  providing  instruction  that  addresses  multiple  learning  styles, 
within  one  lesson.  With  the  advanced  technological  capabilities  of  the  World  Wide  Web,  it  is  possible  for  students 
to  have  exposure  to  a variety  of  instructional  delivery  methods  that  can  accommodate  multiple  learning  styles. 

World  Wide  Web  Technology 

Some  current  capabilities  of  Web  development  software  and  World  Wide  Web  technology  have  allowed 
teachers  to  expand  upon  traditional  text  and  lecture  based  classes  by  creating  individualized  lessons  that  are  aligned 
with  teachers'  lesson  plans  and  individual  students’  competencies  and  abilities.  Some  current  Web  development 
software  tools  include,  Microsoft  Front  Page,  Macromedia  Flash,  Hyper  Studio,  Claris  Home  Page,  and  Power 
Point.  This  Web  development  software  also  allows  each  student  to  meet  his  or  her  individual  needs,  by  selecting  a 
unique  course  of  instruction,  through  non-linear,  branching  structures.  Students  can  direct  their  own  learning  by 
pursuing  their  interests  and  organizing  and  synthesizing  data  and  constructing  projects  that  are  meaningful  and  can 
be  applied  to  real  life  situations.  Ayersman  and  Minden  (1995)  state  that  hypermedia  has  the  ability  to  deliver 
information  in  contextually  meaningful  sequences,  at  a variable  pace  controlled  by  the  learner,  through  multiple 
sensory  modalities.  Summerville  (1997)  states  that  hypermedia  holds  great  promise  for  the  accommodation  of 
individual  differences.  Hypermedia  has  the  ability  to  be  flexible  or  structured,  provide  varied  feedback,  and  allow 
the  user  to  access  other  resources. 

Some  of  the  current  features  in  multimedia  Web  development  include  the  use  of  audio,  video,  virtual 
reality,  animation,  and  simulation.  RealAudio  provides  music,  news,  and  talk  shows  over  the  World  Wide  Web.  Java 
is  a programming  language  that  lets  Web  page  developers  add  software  applications,  games,  animation,  and  other 
features  to  Web  pages.  Shockwave  enables  the  playback  of  high-impact  multimedia  on  the  Web  and  Virtual  Reality 
Modeling  Language  (VRML)  describes  how  three-dimensional  scenes  are  delivered  across  the  World  Wide  Web 
(Serim&Koch,  1996). 

The  World  Wide  Web  can  be  used  to  accommodate  students  with  diverse  styles  in  processing  sensory 
information,  such  as  visual,  auditory,  and  kinesthetic  learners  (Ross,  & Schulz,  1999).  Visual  students  can  benefit 
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from  World  Wide  Web  instruction  that  includes  course  animations,  hypertext,  or  clickable  diagrams  and  video  clips. 
These  images  can  clarify  concepts  that  static  textbooks  cannot.  Learners  who  have  difficulty  processing  auditory 
information  in  a lecture  could  benefit  from  having  the  professor's  lecture  notes,  slide  presentations,  or  overhead 
slides  online.  Auditory  learners  can  benefit  from  Web-based  instruction  by  having  professors  record  their  lectures, 
record  class  summaries,  or  create  archived  sound  resources  and  place  them  online  (Ross,  1998).  Kinesthetic  learners 
can  benefit  from  Web-based  instruction  by  providing  them  with  Java  based  puzzles,  games  and  simulations. 
Providing  a course  listserv,  bulletin  board,  discussion  thread,  or  chat  room  can  also  accommodate  collaborative 
learners  (Ross  & Schultz,  1999). 

Lin  and  Davidson  (1996)  found  that  when  using  hypertext  instruction,  structure  and  cognitive  style  had  a 
significant  effect  on  student  performance.  Research  has  shown  that  learner  control  is  an  important  factor  when 
creating  Web-instruction  for  diverse  learning  styles.  Learner  control  is  the  amount  of  personal  responsibility  an 
individual  can  have  in  an  instructional  lesson.  Learner  control  in  Web-based  instruction  can  be  exhibited  through 
sequencing,  pacing,  advisement  level,  practice,  and  amount  of  material  (Rasmussen  & Davidson-Shivers,  1998).  In  a 
study  conducted  by  Rasmussen  and  Davidson-Shivers  (1998),  the  influences  of  the  individual  differences  in  learning 
styles  and  the  concept  of  learner  control  to  assist  instructors  and  developers  in  designing  effective  instruction  for  all 
learners  were  investigated.  They  hypothesized  that  learning  styles  can  be  used  in  conjunction  with  learner  control  to 
facilitate  and  enhance  student  performance  in  hypermedia  learning  environments.  The  study  found  that  learning 
styles  significantly  influenced  performance  in  hypermedia  learning  environments.  Individuals  who  had  a learning 
style  preference  towards  active  learning  preferred  low  levels  of  learner  control  and  performed  best  in  the  hierarchy 
structures.  The  hierarchical  structures  allowed  students  to  quickly  accomplish  tasks,  which  fits  the  style  of  the  active 
learner  who  prefers  to  complete  tasks  as  quickly  as  possible.  Individuals  who  were  reflective  in  their  learning  style 
preference  performed  highest  with  moderate  structure.  The  results  show  that  reflective  learners  must  have  some 
structure  but  must  also  be  provided  with  the  ability  to  explore  other  related  material.  In  addition,  all  learners 
performed  well  in  an  environment  that  provided  moderate  structure.  This  research  suggests  that  environments  with 
high  levels  of  learner  control  may  be  counterproductive  to  all  learning  style  types. 

World  Wide  Web  and  Learning  Styles 

World  Wide  Web  technology  has  the  capability  of  meeting  the  individual  needs  of  a variety  of  learner  types 
within  each  Web-based  lesson.  Much  of  the  instruction  taught  in  schools  benefits  the  auditory  and  visual  learners. 
Other  learning  style  types  are  often  not  focused  on  in  the  classroom.  Although  research  on  Web-based  multimedia 
instruction  and  learning  styles  is  extremely  limited,  the  following  examples  have  been  found  that  support  the  use  of 
Web-based  multimedia  instruction  for  meeting  the  diverse  needs  of  learners.  For  example,  tactile  learners  gain  and 
attain  information  when  they  are  given  hands-on  activities  (Dunn  & Dunn,  1978).  These  students  can  respond  well 
to  game-like  activities  that  are  naturally  motivating  and  self-correcting.  Students  can  use  the  self-correcting  features 
to  discover  correct  answers  through  inductive  and  deductive  learning  (Bruno,  1982).  An  example  of  a game  activity 
that  can  be  beneficial  to  a tactile  learner  is  the  microworld.  A microworld  is  an  exploratory  learning  environment, 
discovery  space,  and  constrained  simulation  of  real-world  phenomena  in  which  learners  can  navigate,  manipulate  or 
create  objects,  and  test  their  effects  on  one  another  (Jonnassen,  1996).  A study  conducted  by  Stoney  & Oliver 
(1999),  explored  the  notion  that  students  who  learn  in  an  applied  setting,  such  as  a microworld,  will  experience 
cognitive  engagement  and  motivation  through  the  relevance  of  the  material  to  the  student's  real  world.  University 
students  were  selected  and  their  activities,  communication,  and  interactions  were  observed.  Results  found  that  the 
use  of  well-designed  interactive  microworlds  leads  to  learner  cognitive  engagement  and  will  drive  learners  towards 
greater  levels  of  higher  order  thinking.  The  microworld  game  also  provided  motivation  and  engagement  with  the 
program  content  because  it  provided  real  life  experience.  The  game  helped  learners  to  judge  and  assess  the 
credibility  of  potentially  conflicting  information  and  to  develop  strategies  to  think  critically,  resolve  conflicts,  and 
solve  current  and  potential  problems. 

The  California  State  University  Biology  Labs  On-Line  Project  (Bell,  1999)  is  a project  that  seeks  to  use 
technology  for  improving  biology  education.  This  project  provides  Web-based  simulations  for  different  Biology  labs 
in  which  learners  can  access  anywhere  and  anytime.  Initial  tests  on  the  Biology  project  have  suggested  that  the 
programs  can  be  very  useful  learning  tools.  Students  liked  the  way  the  simulations  made  them  think,  solve  problems, 
and  understand  the  breadth  of  the  material  better.  The  one  disadvantage  noted  was  that  novice  computer  users 
require  assistance  to  get  the  most  out  of  the  simulations. 

An  additional  case  study  involving  a college  Engineering  Graphics  class,  at  the  University  of  Texas,  has 
used  the  World  Wide  Web  to  publish  a comprehensive  multimedia  instructional  CD-ROM  Web  page  (Crown, 
1999).  The  CD-ROM  Web  page  consists  of  an  integrated  web  site  with  links  to  hours  of  tutorial  movies,  lecture 
presentations,  web-based  games  that  reinforce  course  topics,  and  interactive  web-based  quizzes.  The  CD-ROM  Web 
page  was  designed  to  make  more  efficient  use  of  faculty  time  and  to  provide  students  with  additional  individualized 
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help.  This  project  had  many  positive  results  on  both  faculty  and  students.  Professors  benefited  by  having  their  course 
lecture  time  reduced  by  eighty  percent.  Students  who  require  extra  help  or  need  to  make  up  work  were  able  to  catch 
up  and  perform  well  in  the  course.  The  visualization  tools,  provided  in  the  games,  offered  students  the  opportunity  to 
view  objects  repeatedly,  which  would  be  difficult  to  teach  using  other  methods.  Two  noted  drawbacks  to  the  CD- 
ROM  Web  page  include  the  high  initial  cost  in  time  to  develop  the  project  and  the  fact  that  this  environment  has  less 
student-to- student  interactivity  and  is  more  impersonal  than  traditional  classroom  lab  settings.  The  most  notable 
changes  in  student  performance  were  by  those  students  that  fell  outside  the  average  range.  The  advanced  students 
were  stimulated  and  motivated  by  the  self-paced  nature  of  the  course  and  the  students  who  found  the  material 
challenging  were  able  to  work  at  a slower  pace  and  finish  a course  that  they  normally  would  drop. 

Conclusion 

Little  Research  has  been  done  on  addressing  multiple  learning  styles  in  Web-based  instruction.  The 
literature  shows  strong  support  for  matching  learning  activities  with  students'  preferred  learning  styles.  Web-based 
instruction  allows  for  the  development  of  instruction  that  can  meet  a variety  of  learning  styles.  It  can  address 
multiple  learning  styles  within  the  same  instructional  lesson.  Therefore,  it  is  hypothesized  that  students  will  perform 
better  and  learn  more  from  properly  designed  Web-based  instruction  that  meets  the  many  individual  learning  styles 
of  students,  than  Web-based  instruction  that  is  linear  and  text  based.  Since  World  Wide  Web  technology  and 
multimedia  software  is  capable  of  supporting  instruction  that  appeals  to  a variety  of  learning  styles,  and  research 
suggests  that  students  must  also  be  exposed  to  other  non-dominant  learning  styles,  a Web-based  instructional  lesson 
that  accommodates  multiple  learning  styles  should  be  ideal  for  enhancing  student  achievement.  When  addressing  the 
no  significant  difference  phenomena,  the  research  presented  in  this  literature  review  supports  that  there  are 
significant  differences  in  student  achievement,  in  a Web-based  environment,  when  instruction  is  developed  using 
theories  on  the  way  students  learn.  In  order  to  look  beyond  the  no  significant  difference  phenomena,  Web  instruction 
must  go  beyond  the  technology  and  look  into  how  the  technology  can  be  used  to  deliver  instruction  to  all  students, 
each  unique  in  their  experiences  and  in  the  ways  they  learn.  Integrating  learning  styles  research  into  Web-based 
instruction  is  one  way  that  may  show  a significant  difference  in  achievement  in  Web-based  instruction. 

Application  of  Intelligent  Agents  in  Web-based  Learning  Systems 

How  will  the  Intelligent  Agents  (IAs)  and  World  Wide  Web  (WWW)  architecture  impact  education  in  the 
future?  Perhaps  learning  will  resemble  this  imaginary  encounter: 

The  twelve-year-old  girl  enters  a room  containing  a wall  display  and  an  electronic  unit  about  the 
size  of  the  obsolete  desktop  PCs  of  a decade  ago.  Her  “learning  center”  is  a voice  activated  intelligent 
appliance  linked  directly  to  a database  via  the  new  Information  Superhighway. 

“Computer  on, " she  commands. 

The  computer  synthesized  voice  responds,  Good  morning  Genevieve.  School  was  to  begin  27 
minutes  ago.  Are  you  ready? 

“Yes,”  she  replies  rather  unenthusiastically. 

/ recommend  beginning  with  Statistics.  You  are  now  one  week  behind  your  agreed  schedule , the 
course  management  agent  continues. 

“Great!,”  the  girl  responds  with  even  less  enthusiasm  than  before. 

I'm  sorry.  Is  that  an  affirmative  Genevieve ? 

“Yes” 

The  display  screen  next  shows  the  animated  face  of  a dog  wearing  glasses  (the  girl’s  creation). 
Genevieve,  good  to  see  you.  Your  reflective  time  is  over.  Are  you  ready  to  review  your  last  exam?,  the 
animated  persona  smiles  with  lolling  tongue  awaiting  the  girl’s  voice  response. 

“Not  really,  but  OKAY  Topper”  (The  girl’s  name  for  her  animated  tutor).  She  smiles  as  she 
emphasized  the  command  word  “okay.” 

You  completed  30  of  50  questions  correctly,  Topper  continues. 

“I  figured  as  much” 

The  animated  persona  looks  quizzical  and  responds,  Yes.  Well,  it  appears  that  you  are  having 
difficulty  relating  the  concept  of  the  normal  distribution  to  the  concepts  of  standard  deviation  and 
confidence  intervals.  Do  you  want  to  review  the  material  or  try  a new  approach. 

“I’ll  try  the  new  approach.”  And  the  lesson  begins. 


At  what  point  in  the  future  will  technology  and  software  combine  to  produce  the  scene  above?  As  prototypes  and 
experimental  models,  this  future  is  already  here. 

Alan  Turing  and  Moore’s  Law 

In  a 1950  article,  Computing  Machinery  and  Intelligence,  mathematician  Alan  Turing  posited  the  question, 
"Can  a Machine  Think?"  (Hodges,  2000).  To  Turing  the  question  was:  "If  a computer  could  think,  how  could  we 
tell?"  He  considered  that  if  the  computer’s  responses  were  indistinguishable  from  that  of  a human,  could  the 
computer  be  said  to  be  thinking.  Turing  set  a standard  for  determining  intelligence  or  thinking  by  making  a 
prediction.  He  postulated  that  within  fifty  years  (i.e.,  the  year  2000)  an  average  person  would  not  have  more  than  a 
70  percent  chance  of  distinguishing  between  a computer  or  human  responding  to  five  minutes  of  questioning. 
Turing’s  imitation  game  has  now  become  known  as  the  Turing  Test.  In  1990  philanthropist  Hugh  Loebner  agreed  to 
underwrite  a contest  to  implement  the  Turing  Test.  The  contest  is  conducted  each  year  by  The  Cambridge  Center 
for  Behavioral  Studies  ( http://www.loebner.net/Prizef/loebner~prize.htmB.  A grand  prize  of  $100,000  and  a Gold 
Medal  will  be  awarded  to  the  first  person  constructing  a computer  that  passes  the  Turing  Test.  Although  this  prize 
has  yet  to  be  claimed,  each  year  an  prize  of  $2000  and  a bronze  medal  is  awarded  to  the  individual  constructing  a 
computer  program  offering  the  most  “human”  responses. 

In  addition  to  software  advances  required  to  pass  the  Turing  Test  technology  must  also  progress.  Moore’s 
Law  is  based  on  the  observation  that  the  logic  density  of  silicon  integrated  circuits  follows  a mathematical  curve. 
According  to  its  author  Gordon  Moore  (who  co-founded  Intel)  the  amount  of  information  storable  on  a given 
amount  of  silicon  has  roughly  doubled  every  year  since  the  technology  was  invented.  In  the  1970s  the  rate  of 
doubling  slowed  to  18  months,  and  Moore  predicted  that  natural  limitations  would  invalidate  the  “Law,”  but  not 
until  2017  (Kanellos,  1997). 

In  a broader  view  both  the  Turing  Test  and  Moore’s  Law  are  illustrative  of  the  first  steps  toward  an  ideal 
where  economical  intelligent  machines  serve  the  needs  of  the  individual.  This  paper  will  examine  the  development 
of  intelligent  agents  and  forecast  applications  for  education  deliverable  on  the  World  Wide  Web  (WWW). 

Intelligent  Agents 

‘Intelligent  agent’  as  a concept  has  been  around  for  about  25  years.  An  agent  is  a software  entity  that  has 
some  degree  of  autonomy,  carries  out  operations  on  the  behalf  of  a user  or  another  program,  and  represents  or  has 
knowledge  of  the  user's  goals  and  wishes.  Definitions  approach  intelligent  agents  based  on  how  the  term 
“intelligent”  is  defined.  Science  fiction  genres  tend  to  associate  intelligent  software  with  human-like  emotional  and 
mental  processes  such  as  knowledge,  belief,  intention,  and  obligation  (Coen  1994).  Utilitarian  definitions  of 
intelligent  agents  focus  on  a particular  software’s  ability  to  function  in  some  complex  dynamic  environment,  and  to 
sense  and  react  autonomously  to  achieve  a set  of  behaviors.  Intelligent  agents  can  also  simulate  worlds  and  operate 
in  those  simulated  worlds  interacting  with  the  users  and  other  agents.  Information  agent  software  searches  multiple 
databases  to  retrieve,  collate,  filter,  and  organize  information  to  answer  queries  from  users  (Shoham,  1993).  A list 
of  attributes  of  agents,  not  all  of  which  need  to  be  present  in  an  agent,  are: 

• Autonomy  - ability  to  operate  without  the  direct  intervention  of  humans; 

• Social  ability  - can  interact  with  other  agents  or  humans  by  providing  assistance  to  users  dealing  with  another 
agent; 

• Reactivity  - have  perception  of  their  environment  which  allows  timely  response  to  changes  that  occur  in  it; 

• Proactivity  - exhibit  goal-directed  behavior  (initiative); 

• Mobility  - can  move  to  other  environments; 

• Reusability  - agent  algorithms  run  continuously; 

• Adaptivity  - will  automatically  adapt  to  changes  in  their  environment;  and 

• Synergism  - higher  level  of  interoperability  is  made  possible  through  the  interaction  of  agents  and  humans  as  a 
system  (Shoham,  1993). 

There  are  many  categories  that  research  on  intelligent  agent  systems  and  similar  software  applications  have 
been  subdivided.  For  example,  adaptive  learning  systems,  artificial  intelligence,  artificial  life,  biocybemetics, 
cognitive  and  neural  modeling,  evolutionary  computation,  fuzzy  systems,  genetic  algorithms,  knowledge-based 
systems,  multi  agent  systems,  neural  networks,  parallel  and  distributed  computing,  self-organizing  systems  are  terms 
associated  with  the  above  attributes  for  intelligent  agents.  Furthermore,  each  of  these  categories  is  being  researched 
for  applicability  to  learning  on  the  WWW. 

Intelligent  Agents  and  Tutoring  Systems  on  the  WWW 

With  the  World  Wide  Web  becoming  an  increasingly  important  platform  for  the  delivery  of  educational 
content,  instructional  designers  and  researchers  in  artificial  intelligence  are  reconsidering  architectures  that  were 


developed  for  stand-alone  computer  or  local  network  applications  (Ritter,  1997).  Intelligent  agents  are  being 
developed  for  many  of  the  new  applications  (Johnson  & Shaw,  1997).  Areas  of  intelligent  agent  research  that  show 
promise  for  educational  applications  deliverable  through  the  WWW  are  briefly  described  in  this  paper. 

Intelligent  Agents  Used  for  Database  Management  and  Information  Retrieval 

The  WWW  can  be  viewed  as  an  extremely  large  database.  This  database  is  composed  of  heterogeneous 
documents  that  are  accessed  via  a wide-area  network  (WAN)  and  a client-server  protocol.  These  documents  (nodes) 
are  connected  by  hyperlinks.  To  retrieve  information  hyperlinks  must  be  navigated  and  navigation  becomes 
extremely  difficult  in  the  three-dimensional  structure  of  the  WWW  (De  Bra  & Post,  1994). 

Intelligent  agents  have  the  potential  to  search  the  vast  volume  of  information  on  the  Web  in  ways  that  are 
only  now  being  explored.  Agents  have  advantages  of  (1)  robustness  - available  24  hours  a day,  seven  days  a week, 
(2)  speed  - transmit  information  within  seconds,  and  (3)  value  adding  - the  more  sophisticated  the  agent  in  terms  of 
design  for  ability  to  discover  relevant  information  the  more  tangible  the  benefit  from  the  information  (Etzioni, 
1999).  Etzioni  and  colleagues  at  the  University  of  Washington  are  researching  and  designing  Internet  Softbots 
(intelligent  agent  software)  that  will  enable  a user  to  specify  personalized  wants.  The  Softbot  is  designed  to 
calculate  the  how  and  where  to  satisfy  the  request.  Etzioni  uses  the  metaphor  of  a food  chain  or  pyramid  to  visually 
describe  the  role  that  a discriminating  information  agent  would  play  in  this  new  information  “ecosystem”  of  the 
WWW.  Developmental  work  is  also  being  done  by  Etzioni’s  group  on  adaptive  Web  sites  that  automatically 
reconfigure  their  layout  and  presentation  by  analyzing  user  access  patterns  recorded  in  their  server  logs  (Perkowitz 
& Etzioni,  1997). 

Intelligent  Agents  and  User-Friendly  Interfaces  for  Databases 

The  use  of  an  animated  persona  to  communicate  information  has  recently  caught  the  interest  of  the  news 
media.  The  highly  publicized  virtual  newscaster  Ananova  (Hopper,  2000)  is  an  example.  Using  streaming  video 
Ananova  provides  the  latest  headline  news.  Facial  expressions  are  cued  by  XML  (extensible  Markup  Language) 
tags  that  are  added  to  the  script.  Digital  Animations  Online  (http://www.digital-animations.co.uk/)  is  the  creator  of 
Ananova. 

Customizable  animated  intelligent  agents.  Customizable  animated  intelligent  agents  are  available  to  Web 
developers  to  enhance  the  commercial  value  of  a Web  site.  These  virtual  personalities  can  assist  a visitor  to  the  site 
in  identifying  products,  guiding  the  visitor  through  product  features  and  benefits,  or  even  making  the  “sales  pitch” 
and  closing  the  sale.  The  animated  persona  can  interact  with  the  user  by  answering  queries  including  the  display  of 
facial  features  ascribed  to  various  human  emotions.  Animated  personas  are  proposed  by  their  developers  as 
adaptable  for  use  by  educators. 

Intelligent  Agents  Facilitating  Collaboration  in  Online  Courses 

A significant  problem  experienced  by  students  (as  well  as  causing  concern  for  instructional  designers  of 
online  courses)  is  how  to  facilitate  collaboration  in  a largely  asynchronous  environment.  Typical  problems 
expressed  by  students  working  as  groups  in  a face-to-face  environment  are  “lack  of  time,  lack  of  skills  and  members 
not  contributing”  (Whatley,  Staniford,  Beer  & Scown,  1999).  These  problems  are  made  more  manifest  in 
collaborating  groups  separated  geographically  (Johnson,  Aragon,  Shaik,  & Palma-Rivas,  2000).  In  order  to  facilitate 
collaboration  intelligent  agents  are  able  to  support  planning  through  task  allocation,  progress  monitoring,  and 
problem  flagging  (with  communication  with  the  individual  student  as  well  as  with  the  agents  of  other  students). 

O’Riordan  and  Griffith  describe  an  multi-agent  system  design  where  several  intelligent  agents  that  the 
authors  believe  will  overcome  specific  web-based  education  shortcomings  in  the  areas  of  peer-peer  learning,  static 
content,  and  personalized  learning  (O’Riordan  & Griffith,  1999).  Their  system  design  focuses  on  a User  Modeling 
Agent  that  maintains  the  student  profile.  The  profile  consists  of  material  covered  and  how  fast  its  was  covered,  quiz 
score,  frequency  of  links  visited,  and  use  of  an  FAQ  section.  Collaborative  groups  are  based  on  multiple  profiles. 

Intelligent  tutoring  systems.  An  Intelligent  Tutoring  System  (ITS)  is  designed  to  simulate  what  occurs 
between  a student  and  teacher  one-to-one.  ITSs  typically  have  four  components:  (1)  an  intelligent  interface 
(communication),  (2)  a knowledge  domain,  (3)  an  instructional  module,  and  (4)  a student  model  (representing  the 
student’s  current  state  of  knowledge).  Student  modeling  has  been  the  most  difficult  due  to  the  learner  uniqueness 
and  individual  learning  styles  (Stem,  1 997). 

Most  models  of  student  behavior  commonly  used  with  intelligent  tutoring  systems  (ITSs)  or  not  designed  to 
deal  with  such  human  inputs  as  inconsistency,  incompleteness,  and  ambiguity.  Often  this  “fuzziness”  of  human 
input  results  in  an  inability  of  the  ITSs  to  correctly  evaluate  student  answers.  Huang  approached  this  problem  by 
developing  a model  that  incorporates  fuzzy  set  theories  and  two-dimensional  (Hasse)  diagrams  to  provide  more 
accurate  feedback  to  incomplete  student  answers  (Huang,  1999).  The  learning  agent  captures  inconsistent  behaviors 
of  the  student.  A student’s  inconsistent  response,  for  example,  triggers  the  learning  agent  procedure  that  uses  a 
inconsistency  identifier.  The  inconsistency  identifier  is  based  on  a heuristic,  such  as  on  the  same  or  similar  question 
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the  student’s  answer  is  now  incorrect.  The  feedback  from  the  learning  agent  then  displays  features  of  the 
inconsistencies  and  provides  suggested  strategies  to  prevent  the  behavior  from  happening  again. 

Animated  pedagogical  agents . Animated  pedagogical  agents  are  designed  to  facilitate  learning  in 
computer-based  instruction,  and  also  have  been  applied  to  Web-based  intelligent  tutoring  systems  (Rickel  & 
Johnson,  1997).  Typically,  animated  agents  have  personas  that  can  exhibit  emotive-kinesthetic  behaviors 
responding  to  user  input  within  the  program  (Lester,  Stone  & Stelling,  1999).  Pedagogical  agents  are  built  upon  ITS 
research,  but  have  the  added  requirements  of  coordinating  the  agents  behavior  with  personality  cues  that  relate  to 
student  input  (Lester,  Towns  & FitzGerald,  1999).  Animated  agents  can  provide  real-time  advice  that  supports  a 
learners’  problem  solving  activity,  but  can  also  exhibit  contextually  appropriate  emotive  behaviors  such  as  facial 
expressions  and  gestures.  Lester,  et  al  identify  two  key  problems  with  representing  emotive  behaviors  are 
conversational  believability  and  emotive  believability,  and  have  developed  a dialogue  mapping  framework  to 
achieve  conversational  believability  and  full-body  emotive  behaviors  in  response  to  learner  problem  solving 
activities  to  achieve  increased  emotive  believability. 

An  example  of  a prototypical  animated  intelligent  agent  is  was  developed  by  Andre,  Rist,  & Muller  (1997). 
Their  agent  did  not  involve  speech,  but  with  animated  gestures  and  text  interacted  with  the  user.  One  unique  feature 
of  their  animated  agent  was  that  the  presentation  scripts  and  navigation  structures  were  not  storied,  but  generated 
just-in-time  from  pre -authored  document  fragments. 

Adele  (Agent  for  Distance  Education)  is  an  example  of  a more  sophisticated  animated  pedagogical  agent 
developed  by  researchers  at  the  Center  for  Advanced  Research  in  Technology  for  Education  (Shaw,  Johnson  & 
Ganeshan,  1999).  Adele  was  specifically  designed  to  support  online  students  in  problem-solving  exercises  with  a 
simulated  patient  in  a clinical  setting.  Students  are  able  to  ask  questions  regarding  the  simulated  patient’s  medical 
history,  perform  a physical  examination,  order  diagnostic  tests,  and  make  diagnoses.  Adele  also  provides  the 
student  feedback  relative  to  the  actions  chosen  by  the  student  in  the  form  of  suggested  correct  actions,  or  hints  and 
rationales  for  particular  actions  with  reference  to  relevant  background  material.  In  this  autonomous  agent  design, 
Adele  consists  of  two  sub-components:  the  animated  persona  (a  Java  applet)  and  the  reasoning  engine.  The 
reasoning  engine  performs  all  monitoring  and  decision  making  activities.  Decisions  are  based  on  a student  model, 
the  case  design,  and  the  initial  state  of  the  case.  All  this  information  downloaded  from  a server  when  as  the  student 
selects  the  case.  Initial  student  evaluations  of  Adele  concluded  that  the  hints  and  rationales  provided  by  the 
animated  agent  were  helpful,  but  not  complete.  However,  this  appeared  to  be  a function  of  the  students  not 
interacting  with  the  persona,  for  example,  asking  “Why?”  The  agent  was  capable  of  accessing  more  authored 
knowledge.  This  problem  was  remediated  by  allowing  the  agent  to  provide  more  of  an  explanation. 

Conclusion 

The  significance  of  intelligent  agent  systems  on  the  WWW  will  probably  eventually  be  best  understood  in 
the  context  of  a paradigm  shift.  The  combination  of  technology  in  terms  of  software  and  hardware,  and  the  delivery 
and  accessibility  afforded  by  the  WWW,  is  creating  an  environment  of  change.  The  impact  of  change  on 
institutional  or  traditional  education  is  in  the  infancy  of  exploration. 

Web-based  educational  systems  represent  an  area  that  growing  exponentially  with  educational  technology 
research  and  software  development  leading  the  way.  Pedagogical  aspects  such  as  instructional  design  theories  and 
models  have  lagged  behind  (Tergan,  1997).  In  many  respects  the  situation  facing  educators  is  similar  to  the 
company  that  designs  a product  and  goes  out  to  find  a market  for  it.  The  technology  is  driving  the  process  and 
instructional  theory  and  design  are  attempting  to  “find  the  market.” 

This  situation  may  be  due  to  the  fact  that  much  of  the  developmental  research  work  on  Web-based  IA 
software  being  done  for  commercial  objectives.  Major  efforts  in  IA  research  and  design  are  directed  toward  e- 
commerce  applications  on  the  WWW.  Software  agent  technologies  are  being  investigated  to  expedite  the  electronic 
commerce  revolution.  E-commerce  development  issues,  such  as  distributed  component-based  marketplaces, 
protocols  for  locating  and  defining  goods  and  services,  value-based  product  comparisons,  buying  decision  aids, 
visualization  of  marketplace  data  and  activities,  have  pedagogical  counter-parts  in  education. 

Intelligent  agent  systems  that  support  student  collaboration,  support  problem  solving,  and  explain  errors 
will  certainly  be  a first  step.  How  will  Web-based  distance  education  benefit  from  this  development?  The 
development  of  quality  databases  on  the  WWW  coupled  with  the  development  of  intelligent  agent  software  that  can 
retrieve  personalized  information.  Will  a student  having  greater  access  to  knowledge  be  a better  learner?  What 
instructional  designs  will  be  necessary?  Are  animated  personas  described  in  this  paper,  that  guide  a student  through 
a lesson  or  topic,  an  advantage  to  learning  or  a distraction?  Opportunities  abound  for  the  researcher  in  both  the 
design  and  in  the  development  of  instructional  models  that  incorporate  intelligent  agents  and  the  WWW. 
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Summary 

The  four  approaches  presented  here  support  the  need  for  continued  exploration  of  distance  education  from 
new  perspectives.  The  use  of  CAI  is  not  new.  However,  the  use  of  CAI  as  the  technology  of  choice  for  distance 
education  in  the  K-12  setting  has  potential.  CAI’s  capabilities  of  individualizing  instruction  and  providing 
asynchronous  instruction  combined  with  the  richness  of  the  World  Wide  Web  need  further  exploration  as  an 
effective  and  cost  efficient  means  of  providing  access  to  educational  opportunities  for  K-12  students. 

More  studies  into  learning  styles  of  distance  learners  are  needed  that  control  for  the  problems  identified  in 
the  current  literature  in  this  area.  Sampling  needs  to  be  moved  from  simply  relying  on  convenience  sampling.  In 
addition,  standardized  instruments  need  to  be  developed  to  examine  this  learner  attribute.  Designer’s  of  distance 
education  need  to  address  the  needs  of  the  dependent  learner  and  student  support  services  developed  to  assist  these 
learners  in  having  success  in  the  distance  education  environment.  When  reviewing  learning  styles  and  web-based 
instruction,  there  appears  to  have  been  little  research.  From  a design  perspective  can  and  should  web-based 
instruction  be  designed  in  such  a way  to  accommodate  all  learning  styles? 

New  technologies  are  constantly  emerging.  How  will  the  use  of  intelligent  agents  impact  the  learning 
environment  particularly  distance  education?  Instructional  technology  has  a tendency  to  lag  behind  technology 
advancements.  Exploring  the  utilization  of  intelligent  agents  to  support  student  collaboration,  problem  solving  and 
provide  assistance  in  the  distance  learning  environment  would  be  one  step  toward  closing  the  gap. 

Certainly,  thinking  of  distance  education  research  only  in  terms  of  the  no  significant  difference  does  not 
represent  the  wealth  of  research  that  is  occurring  in  the  field.  However,  as  researchers  we  can  do  more  by  looking 
beyond  the  conventional,  looking  at  new  uses  for  current  technologies,  and  exploring  the  potential  of  emerging  and 
future  technologies. 


References 

Abrioux,  D.  (1991).  Computer- Assisted  language  learning  at  a distance:  An  international  survey. 
TheAmerican  Journal  of  Distance  Education,  5 (1),  3-14. 

Andre,  E.,  Rist,  T.  & Muller,  J.  (1997).  WebPersona:  a life-like  presentation  agent  for  educational 

applications  on  the  World-Wide  Web.  Intelligent  Education  Systems  on  the  World  Wide  Web.  8^  World 
Conference  of  the  AIED  Society,  Kobe,  Japan,  18-22  August.  Retrieved  March  20,  2000  from  the  World  Wide 
Web:  http:/Avww.contrib. andrew.cmu.edu/~plb/AIED97  workshop/Andre/Andre.html 

Ayersman,  D.J.  & Minden,  A.V.  (1995).  Individual  differences,  computers,  and  instruction.  Computers  in 
Human  Behavior,  1 1 (3-4),  371-390. 

Barker,  B.  O.  & Dickson,  M.  W.  (1996).  Distance  learning  technologies  in  K-12  schools:  Past,  present, 
and  future  practice.  TechTrends,  41  (6),  19-22. 

Baynton,  M.  (1992).  Dimensions  of  "control"  in  distance  education:  A factor  analysis.  The  American 
Journal  of  Distance  Education,  6 (2),  17-31. 

Becker,  D.,  and  Dwyer,  M.  (1998).  The  impact  of  student  verbal/visual  learning  style  preference  on 
implementing  groupware  in  the  classroom.  Journal  of  Asynchronous  Learning,  2(2),  61  - 69. 

Bell,  J.  (1999).  The  biology  labs  on-line  project:  producing  educational  simulations  that  promote  active 
learning.  Interactive  Multimedia  Electronic  Journal  of  Computer-Enhanced  Learning.  [On-line].  Available:  http:// 
imej.wfu.edu/articles/1 999/2/01 /index. asp 

Brewer,  S.  M.  & Erickson,  D.  R.  (1997).  Tale  of  two  classrooms.  Journal  of  Computing  in  Teacher 
Education,  13  (3),  20-22. 

Bruno,  A.  (1982).  Student  learning  styles  and  brain  behavior.  Reston,  VA:  National  Association  of 
Secondary  School  Principals. 

Ching,  Luk  Suet.  (1998).  The  influence  of  a distance-learning  environment  on  students’  field 
dependence/independence.  The  Journal  of  Experimental  Education,  66(2),  149  - 60. 

Christmann,  D.,  Badgett,  J.,  & Lucking,  R.  (1997).  Microcomputer-based  computer-assisted  instruction 
within  differing  subject  areas:  A statistical  deduction.  Journal  of  Educational  Computing  Research,  16  (3),  281-296. 

Claxton,  C.S.  and  Murrell,  P.H.  (1987).  Learning  styles:  Implications  for  improving  educational  practices. 
ASHE-ERIC  Higher  Education  Report  #4 

CNET  News.com.  (1997,  September  30).  Moore  says  Moore’s  Law  to  hit  wall.  Retrieved  April  5,  2000 
from  the  World  Wide  Web:  http://news.cnet.com/news/0-1003-20Q-322592.html7st.ne.fd.mdh 

Coen,  M.  H.  (1994).  SodaBot:  A software  agent  environment  and  construction  system.  MIT  AI  Lab 
Technical  Report  1493.  Cambridge,  MA:  MIT,  June. 


257 


264 


Coggins,  C.C.  (1988).  Preferred  learning  styles  and  their  impact  on  completion  of  external  degree 
programs.  The  American  Journal  of  Distance  Education,  2(1),  25  - 37. 

Cognition  and  Technology  Group  at  Vanderbilt  (1993).  Anchored  instruction  and  situated  cognition 
revisited.  Educational  Technology.  33.  52  - 70. 

Crown,  S.W.  (1999).  Web-based  Learning:  Enhancing  the  Teaching  of  Engineering  Graphics.  Interactive 
Multimedia  Electronic  Journal  of  Computer-Enhanced  Learning.  [On-line],  Available: 
http://imej.wfu.edU/articles/1999/2/02/index.asp 

Daniel,  J.  I.  (1999).  Computer-aided  instruction  on  the  World  Wide  Web:  the  third  generation.  The 
Journal  of  Economic  Education,  30  (2),  163-74. 

De  Bra,  P.  & Post,  R.  (1994).  Information  retrieval  in  the  World-Wide  Web:  making  client-based  searching 
feasible.  Journal  on  Computer  Networks  and  ISDN  Systems,  27,  183-192. 

Dias,  P.,  Gomes,  M.J.  & Correia,  A.P,  (1999),  Disorientation  in  hypermedia  environments:  mechanisms  to 
support  navigation.  Journal  of  Educational  Computing  Research.  20(2).  93-1  1 7. 

Dunn,  R.  E Dunn,  K.  (1978).  Teaching  students  through  their  individual  learning  styles:  a practical 
approach,  Reston,  VA:  Reston  Pub.  Co. 

Dunn,  R.,  Griggs,  S.A.,  Olson,  J.,  Gorman,  B.,  & Beasley,  M.  (1995).  A meta-analytic  validation  of  the 
Dunn  and  Dunn  learning  styles  model.  The  Journal  of  Educational  Research,  88,  353-362. 

Elliott,  C.,  Rickel,  J.  & Lester,  J.  (1999).  Lifelike  pedagogical  agents  and  affective  computing:  an 
exploratory  synthesis.  Artificial  Intelligence  Today.  Wooldridge  & Veloso  (Eds.),  pp,  195-212,  Berlin:  Springer- 
Verlag, 

Etzioni,  O,  (1999).  Moving  up  the  information  food  chain:  deploying  Softbots  on  the  Web,  Retrieved 
March  23  from  the  World  Wide  Web: 

http://www.cs.washington.edu/research/projects/softbots/www/presentations.html 

Fourie,  I.  (1999),  The  use  of  CAI  for  distance  teaching  in  the  formulation  of  search  strategies.  Mousaion, 
17  (1),  48-75. 

Garcia,  C.E.  (1998).  Exploring  the  use  of  animation  software  with  young  bilingual  students  learning 
science.  Journal  of  Educational  Computing  Research.  19(3).  247-267. 

Gibson,  C.C.  (1996).  Toward  an  understanding  of  academic  self-concept  in  distance  education.  The 
American  Journal  of  Distance  Education,  10(1),  23  -36. 

Gibson,  C.C.  and  Graff,  A.O.  (1992).  Impact  of  adults’  preferred  learning  styles  and  perception  of  barriers 
on  completion  of  external  baccalaureate  degree  programs.  CADE  : Journal  of  Distance  Education/  Revue  de 
l'enseignement  a distance:  7 (1).  Retrieved  February  21,  2000  from  the  Web: 
http://www.icaap.org/joumals/cade/content/vol7. 1/  09__gibson-graff_91  .html. 

Hanson,  D.,  Maushak  N.  J.,  Schlosser,  C.  A.,  Anderson,  M.  L.,  Sorenson,  C.,  & Simonson,  M.R.  (1996). 
Distance  Education:  Review  of  the  literature.  2nd  edition.  AECT. 

Hodges,  A.  (2000).  Alan  Turing:  The  Enigma.  New  York:  Walker  Books.  In  press.  Retrieved  March  20 
from  the  World  Wide  Web:  http://turing.org.uk/book/ 

Hodges,  H.  (1982).  Student  learning  styles  and  brain  behavior.  Reston,  VA:  National  Association  of 
Secondary  School  Principals. 

Hopper,  D.I.  (2000,  April  18).  Virtual  news  jock  Ananova  goes  live.  CNN.  Retrieved  April  18  from  the 
World  Wide  Web: http://www.cnn.eom/2000/TECH/computing/04/l  8/ananova.launch/index.html. 

Huang,  M.  (1999).  A fuzzy  student  modeling  with  two  intelligent  agents.  Journal  of  Educational 
Computing  Research.  21 0).  99-113. 

J ohnson,  S.D.,  Aragon,  S.R.,  Shaik,  N.,  & Palma-Rivas,  N.  (2000).  Comparative  analysis  of  learner 

satisfaction  and  learning  outcomes  in  online  and  face-to-face  learning  environments.  Journal  of  Interactive  Learning 
Research  11(1).  29-49. 

Johnson,  W.L.  & Shaw,  E.  (1997).  Using  agents  to  overcome  deficiencies  in  Web-based  courseware.  In 

Intelligent  Education  Systems  on  the  World  Wide  Web,  8^  World  Conference  of  the  AIED  Society,  Kobe,  Japan, 
18-22  August.  Retrieved  March  21  from  the  World  Wide  Web:  http://www.isi.edu/isd/ADE/papers/aied97/aied97- 
www.htm 

Jonassen,  D.  (1996).  Computers  as  mindtools  for  schools.  Upper  Saddle  River,  NJ:  Merrill. 

Kussmaul,  C.,  Dunn,  J.,  Bagley,  M.,  & Watnik,  M.  (1996).  Using  technology  in  education:  When  and  why, 
not  how.  College  Teaching,  44  (4).  123-126. 


O ^ r 


258 


Lauzon,  A.  C.  & Moore,  G.  A.  B.  (1989).  A fourth  generation  distance  education  system:  Integrating 
Computer- Assisted  Learning  and  computer  conferencing.  The  American  Journal  of  Distance  Education,  3 (1),  38- 
49. 

Lester,  J.,  Stone,  B.  & Stelling,  G.  (1999).  Lifelike  pedagogical  agents  for  mixed-initiative  problem  solving 
in  constructivist  learning  environments.  User  Modeling  and  User-Adapted  Interaction,  9(1-2),  1-44. 

Lester,  J.C.,  Towns,  S.G.  & FitzGerald,  P.J.  (1999).  Achieving  affective  impact:  visual  emotive 
communication  in  lifelike  pedagogical  agents.  The  International  Journal  of  Artificial  Intelligence  in  Education,  10(3- 
4),  278-291. 

Lin,  C-H.  & Davidson,  G.V.  (1996).  Effects  of  linking  structure  and  cognitive  style  on  students' 
performance  and  attitude  in  a computer-based  hypertext  environment.  Journal  of  Educational  Computing  Research, 
15,(4),  317-329. 

Lockard,  J.,  Abrams,  P.  D.,  & Many,  W.  A.  (1997).  Microcomputers  for  twenty-first  century  educators 
(4th  ed.).  New  York:  Longman. 

Martin,  E.  D.  (1993).  Student  achievement  and  attitude  in  a satellite-delivered  high  school  science  course. 
The  American  Journal  of  Distance  Education,  7 (1),  54-61. 

Maxcy,  D.,  Fossey,  F.,  & Maxcy,  S.  J.  (1994).  Distance  education:  Bridging  an  instructional  gap. 

National  Association  of  Secondary  School  Principals,  Reston,  VA.  (ERIC  Document  Reproduction  Service  No.  ED 
374  768) 

Mckenzie,  B.  K.,  Kirby,  E.,  Newbill,  S.,  & Davidson,  T.  J.  (1998,  March).  What  are  the  most  important 
teaching  behaviors  for  distance  instructors?  Perceptions  of  facilitators,  instructors  and  coordinators.  Paper 
presented  at  the  meeting  of  SITE  98:  Society  for  Information  Technology  & Teacher  Education  International 
Conference,  Washington,  DC.  (ERIC  Document  Reproduction  Service  No.  ED  421  093) 

Menczer,  F.,  Willuhn,  W.,  & Belew,  R.K.  (1994).  An  endogenous  fitness  paradigm  for  adaptive 
information  agents.  Retrieved  March  15,  2000  from  the  World  Wide  Web:  http://dollar.biz.uiowa.edu/~fil/IS/info- 
spiders.html 

Russell,  T.  L.  (Ed.).  (1999)No  significant  difference  phenomenon.North  Carolina  State  University, 
O’Riordan,  C.  & Griffith,  J.  (1999).  A multi-agent  system  for  intelligent  online  education.  Journal  of 
Interactive  Learning  Research.  10(3/4).  263-274. 

Olcott,  D.,  Jr.  (1997).  Renewing  the  vision:  Past  perspectives  and  future  imperatives  for  distance 
education.  Journal  of  Continuing  Higher  Education,  45  (3),  2-13. 

Patel,  A.,  Kinshuk  & Russel,  D.  (2000).  Intelligent  tutoring  tools  for  cognitive  skill  acquisition  in  life  long 
learning.  Educational  Technology  & Society,  3(1).  32-40. 

Perkowitz,  M.,  & Etzioni,  O.  (1997).  Adaptive  Web  sites:  automatically  learning  from  user  access  patterns. 
Proceedings  of  the  Sixth  International  WWW  Conference.  Santa  Clara,  CA,  pp.  32-39. 

Powell,  R.,  Conway,  C.,  and  Ross,  L.  (1994).  Effects  of  student  predisposing  characteristics  on  student 
success.  CADE  : Journal  of  Distance  Education/  Revue  de  l'enseignement  a distance:  9(1).  Retrieved  February  21 , 
2000  from  the  Web:  http://www.icaap.org/joumals/cade/content/vol9.  l/8_powell_et_al.html. 

Rasmussen,  K.  & Davidson-Shivers,  G.  (1998).  Hypermedia  and  learning  styles:  can  performance  be 
influenced?  Journal  of  Educational  Multimedia  and  Hypermedia,  7,  (4),  291-308. 

Rasmussen,  K.L.  and  Davidson-Shivers,  G.V.  (1998).  Hypermedia  and  learning  styles:  Can  performance  be 
influenced?  Journal  of  Educational  Multimedia  and  Hypermedia.  7f4).  291-308. 

Richards,  C.N.  and  Ridley,  D.R.  (1997).  Factors  affecting  college  students’  persistence  in  on-line 
computer-managed  instruction.  College  Student  Journal,  31.  490-495. 

Rickel,  J.,  & Johnson,  W.L.  (1997).  Integrating  pedagogical  capabilities  in  a virtual  environment  agent. 
Proceedings  of  the  First  International  Conference  On  Autonomous  Agents,  pp.  30-38. 

Ritter,  S.  (1997).  PAT  Online:  a model-tracing  tutor  on  the  World-wide  Web.  In  Intelligent  Education 

Systems  on  the  World  Wide  Web.  8 World  Conference  of  the  AIED  Society,  Kobe,  Japan,  18-22  August. 
Retrieved  March  20,  2000  from  the  World  Wide  Web: 

http://www.contrib.andrew.cmu.edu/~plb/AIED97  workshop/Proceedings. html, 

Robertson,  I.T.  (1985).  Human  information-processing  strategies  and  style.  Behavior  and  Information 
Technology,  4,  (1),  19-29. 

Robotham,  D.  (1995).  Self-directed  learning:  the  ultimate  learning  style?  Journal  of  European  Inductrial 
Training,  19,  (7),  3-7. 

Ross,  J.L.  & Schulz,  R.A.  (1999).  Using  the  world  wide  web  to  accommodate  diverse  learning  styles. 
College  Teaching,  47.  (4),  123-129. 


259 


266 


Ross,  J.L.  (1998).  On-line  but  off-course:  a wish  list  for  distance  educators.  International  Electronic  Journal 
for  Leadership  in  Learning,  2,  (3). 

Saba  F.  (2000).  Research  in  distance  education:  A status  report.  International  Review  of  Research  in  Open 
and  Distance  Learning,  i (1). 

Serim,  F & Koch,  M.  (1996).  NetLeaming:  why  teachers  use  the  internet.  Sebastopol,  CA:  Songline 
Studios  Inc. 

Shaw,  E.,  Johnson,  W.L.  & Ganeshan,  R.  (1999).  Pedagogical  agents  on  the  Web.  Retrieved  March  28, 
2000  from  the  World  Wide  Web:  http://www.isi.edu/isd/ADE/papers/agents99/agents99.htrn. 

Shoham,  Y.  (1993).  Agent  oriented  programming.  Artificial  Intelligence,  60:51-92. 

Snider,  V.E.  (1990).  What  we  know  about  learning  styles  from  research  on  special  education.  Educational 
Leadership,  48,  (2),  53. 

Stem,  M.K.  (1997).  The  difficulties  in  Web-based  tutoring,  and  some  possible  solutions.  Inlntelligent 

Education  Systems  on  the  World  Wide  Web,  8^  World  Conference  of  the  AIED  Society,  Kobe,  Japan,  18-22 
August.  Retrieved  March  20,  2000  from  the  World  Wide  Web: 
http://www.contrib.andrew.cmu.edu/~plb/AIED97  workshop/Stem. html 

Stoney,  S.  & Oliver,  R.  (1999).  Can  higher  order  thinking  and  cognitive  engagement  be  enhanced  with 
multimedia?  Interactive  Multimedia  Electronic  Journal  of  Computer-Enhanced  Learning.  [On-line].  Available: 
http://imej  .wfu.edu/articles/ 1 999/2/07/index. asp 

Stoyanov,  S.  & Kommers,  P.  (1999).  Agent-support  for  problem  solving  through  concept-mapping.  Journal 
of  Interactive  Learning  Research,  10(3/4),  401-425. 

Summerville,  J.B.  (1997).  The  role  of  awareness  of  cognitive  style  in  hypermedia.  20th  Annual 
Proceedings  of  the  Association  for  Educational  Communications  and  Technology,  USA,  431-444. 

Tergan,  S.  (1997).  Conceptual  and  methodological  shortcomings  in  hypertext/hypermedia  design  and 
research.  Journal  of  Educational  Computing  Research,  16(3),  209-235. 

Trentin,  G.  (1996).  Internet:  Does  it  really  bring  added  value  to  education?  Education  Technology 
Review,  6,  10-13. 

Unz,  D.C.  & Hesse,  F.W.  (1999).  The  use  of  hypertext  for  learning.  Journal  of  Educational  Computing 
Research,  20(3),  279-295. 

Vrasidas,  C.  & Mclsaac,  M.  S.  (1999).  Factors  influencing  interaction  in  an  online  course.  The  American 
Journal  of  Distance  Education,  13  (3),  22-36. 

Wagner,  E.D.  and  McCombs,  B.L.  (1995).  Learner  centered  psychological  principles  in  practice:  Designs 
for  distance  education.  Educational  Technology,  35(2).  32  - 35. 

Whatley,  J.,  Staniford,  G.,  Beer,  M.  & Scown,  P.  (1999).  Intelligent  agents  to  support  students  working  in 
groups  online.  Journal  of  Interactive  Learning  Research,  10(3/4),  361-373. 

Wilkinson,  T.  W.  & Sherman,  T.  M.  (1990).  Perceptions  and  actions  of  distance  educators  on  academic 
procrastination.  The  American  Journal  of  Distance  Education,  4 (3),  47-56. 

Wisher,  R.  A.  & Cumow,  C.  K.  (1999).  Perceptions  and  effects  of  image  transmissions  during  Internet- 
based  training.  The  American  Journal  of  Distance  Education,  13  (3),  37-51 . 

Yang,  Y-C.,  and  Chin,  W-E.  (1996).  Motivational  analyses  on  the  effects  of  type  of  instructional  control 
on  learning  from  computer-based  instruction.  Journal  of  Educational  Technology  Systems.  25(1),  25  - 35. 


260 


267 


Identification  and  Analysis  of  the  Leading  Journals  in  the  Field  of 
Educational  Technology:  A Guide  for  Reading  and  Writing  about 

Educational  Technology 

Nancy  J.  Maushak 
Robert  Price 
Hsi-Chi  Wang 
Texas  Tech  University 


Abstract 

This  article  reports  on  a study  conducted  to  identify  and  analyze  the  content  of  the  five  leading  journals  to 
assist  those  reading,  teaching  and  publishing  in  the  field  of  educational  technology.  Email  was  used  to  contact 
professionals  in  the  field.  They  were  asked  to  identify  the  top  five  journals  in  educational  technology.  Results  were 
summarized  then  the  leading  journals  were  reviewed  for  content  and  number  of  articles.  ETR&D  was  identified  as 
the  leading  journal  in  the  field.  The  content  and  type  varied  from  journal  to  journal  based  on  the  overall  goals  and 
intended  audience  of  the  journal.  Suggestions  for  writing  and  publishing  are  included. 

Introduction 

Research,  teaching  and  service  continue  to  be  the  three  pillars  supporting  recognition,  promotion  and  tenure 
in  the  academic  world.  For  many,  the  greatest  contribution  is  research.  Research,  theory  and  changing  paradigms 
must  be  communicated  to  the  field  in  order  to  make  a positive  contribution.  This  is  as  true  for  educational 
technology  as  it  is  for  other  academic  areas.  Publication  therefore  becomes  a primary  role  for  the  novice  as  well  as 
the  experienced  professor  and  for  the  graduate  students  they  mentor. 

The  process  of  publication  seems  at  times  almost  mystical.  Young  professionals  tend  to  be  intimidated  by 
the  whole  idea,  especially  the  prospect  of  rejection.  They  view  successful  scholars  as  possessing  great  wisdom  and 
intellect  when  in  reality  they  could  be  more  accurately  characterized  as  persistent,  systematic  and,  sometimes,  just 
plain  lucky! 

This  article  provides  a background  of  the  literature  in  educational  technology  related  to  publication  and 
presents  results  of  an  informal  study  which  identified  and  analyzed  the  top  ranked  journals  in  the  field.  This 
information  should  be  of  use  to  both  novice  and  experienced  scholars  and  as  they  share  their  research,  knowledge 
and  expertise  in  the  field  of  educational  technology. 

Background 

Publication  has  always  been  and  continues  to  be  of  great  interest  for  scholars,  researchers,  and  graduate 
students  because  it  is  the  road  to  making  a contribution  to  the  field  and  contributing  to  their  personal  goals  of 
professional  advancement.  A review  of  the  literature  related  to  publishing  in  the  educational  technology  field 
yielded  some  interesting  findings. 

Ninety  percent  of  the  articles  identified  discussed  general  publishing  principles.  Thompson  (1995) 
complied  a what-to-do  list,  and  Newren  (1992)  contributed  a list  of  major  causes  of  rejection  for  writers  to  consider 
before  and  after  they  send  in  their  manuscripts  to  a journal  publisher.  Abelson  (1990),  the  editor  of  the  journal 
Science , talked  about  his  experiences  in  evaluating  submitted  manuscripts.  He  believed  in  the  value  of  peer  review, 
and  tried  to  caution  writers  to  beware  of  any  unethical  behaviors  such  as  plagiarism. 

Newren  (1992),  Thompson  (1995),  Clemente,  Shapiro,  Miheim,  and  Bohlin  (1990)  each  summarized  vital 
information  of  the  many  different  journals  in  the  educational  technology  field.  Types  of  information  presented 
included  contact  information,  acceptance  rate,  readership,  decision  time,  publication  time,  etc.  These  summaries  are 
useful  for  prospective  authors  to  help  them  familiarize  themselves  with  different  journals  in  order  to  more 
effectively  select  a journal  for  manuscript  submission.  Yet,  not  one  of  these  articles  examined  the  types  or  topics  of 
the  articles  actually  published  in  the  journals.  Thompson  (1995)  suggest  that  all  perspective  writers  should  look  into 
the  matter  of  finding  the  types  and  topics  patterns  in  addition  to  identifying  the  guidelines  for  authors  of  the  journal 
they  have  selected  to  send  their  manuscript.  This  step  is  vital  to  the  process  of  getting  a manuscript  accepted  by  the 
specific  journal. 

The  review  of  previous  articles  related  to  scholarly  publication  in  the  field  of  educational  technology 
identified  a need  for  a study  that  identified  information  about  types  and  topics  of  articles  in  the  top  ranked 
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educational  technology  journals  to  assist  scholars  in  selecting  the  best  match  for  their  manuscript  and  to  help  them 
recognize  the  current  trends  and  future  direction  in  educational  technology.  This  study  attempts  to  fill  that  need. 

Methodology 

An  email  survey  was  used  to  identify  the  top  five  journals  being  read  by  educational  technologists. 
Websites  of  nine  schools  with  nationally  recognized  programs  in  educational  technology  were  used  to  identify 
faculty  in  the  field.  An  email  message  was  sent  to  85  faculty  members  asking  them  to  list  in  rank  order  the  top  five 
journals.  In  addition,  they  were  asked  to  respond  with  one  piece  of  advice  for  those  just  starting  to  write  for 
publication. 

An  analysis  of  the  1998-1999  issues  of  the  top  journals  was  conducted  to  identify  the  average  number  of 
articles  per  publication,  general  types  of  articles,  and  topic  categories  of  articles.  Four  categories  for  classification  of 
journal  articles  were  identified  by  Klein  (1997)  in  a review  of  Educational  Technology  Research  and  Development . 
These  categories  included  case  study,  description,  empirical  research,  and  literature  review.  For  this  study,  these 
categories  were  revised  to  include  descriptive,  research,  book  review  and  editorial  articles. 

Topic  categories  of  articles  emerged  from  the  analysis  conducted  by  two  researchers.  Categories  were  not 
exclusive  but  topics  were  placed  under  the  theme  that  best  reflected  the  intent  of  the  article.  Four  broad  themes 
emerged  from  the  review:  design,  environment,  student,  and  teacher.  A fifth  category  (other)  was  used  for 
additional  topics  that  either  encompassed  several  categories  or  seemed  to  fit  none  of  the  emergent  themes. 

Where  possible,  actual  copies  of  the  journal  were  used  for  review  purposes.  The  use  of  on-line  databases 
including  ERIC  and  Education  Abstracts  facilitated  the  review  process. 

Results 

A total  of  30  responses  were  received  to  the  email  survey  with  each  school  represented  by  at  least  one 
faculty  member.  Five  of  the  responses  simply  indicated  the  individual  was  no  longer  in  the  field  or  had  moved  to 
another  institution.  Responses  from  the  remaining  25  (31%  response  rate)  were  summarized  to  identify  the  top  five 
journals.  A total  of  31  different  journals  were  identified.  Educational  Technology  Research  and  Development 
(ETR&D)  was  ranked  first  or  second  by  18  of  those  responding. 

The  Big  Five 

To  identify  the  top  five  journals,  responses  were  assigned  values  based  on  rank.  The  number  one  journal 
listed  by  an  individual  received  a score  of  five  with  others  in  descending  order.  Scores  across  all  responses  were 
summed  and  journals  with  the  highest  values  were  identified.  The  top  five  journals  in  order  with  their  scores  were: 


Educational  Technology  Research  and  Development  9 1 

Performance  Improvement  Quarterly  35 

Educational  Technology  3 1 

Journal  of  Educational  Computing  Research  29 

Instructional  Science  28 

Types  and  Topics 


An  understanding  of  the  number  of  issues  and  number  of  articles  published  on  a yearly  basis  can  be  one 
factor  in  selecting  a journal  for  publication  (Table  1).  Three  of  the  five  top  journals  are  published  quarterly.  In  the 
subscription  information,  Instructional  Science  indicates  it  is  published  six  times  a year.  A review  of  the  actual 
citations  and  abstract  seems  to  indicate  that  while  six  numbers  are  used  for  each  volume,  only  four  actual 
publications  could  be  located.  Numbers  1 and  2,  and  numbers  3 and  4 for  both  1998  and  1999  were  found  to 

be  combined  into  one  issue. 

All  articles  including  introductions  to  special  issues  and  book  reviews  were  counted.  The  average  number 
of  articles  per  year  per  journal  varied  from  a low  of  25  to  a high  of  62  (Table  1).  Similarly,  the  average  number  of 
articles  per  issue  varied  from  just  under  six  for  Journal  of  Educational  Computing  Research  to  almost  10  for 
Educational  Technology . 
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Table  I:  Summary  of  Average  Number  of  Articles  per  Issue  of  the  Top  Five  Journals. 


Journal 

Number 
of  issues 
per  year 

1998 

articles 

1999 

articles 

Average 
per  issue 

Educational  Technology  Research  and  Development 

4 

36 

40 

9.5 

Performance  Improvement  Quarterly 

4 

37 

40 

9.6 

Educational  Technology 

6 

57 

62 

9.9 

Journal  of  Educational  Computing  Research 

8 

49 

43 

5.8 

Instructional  Science 

4(6)* 

25 

31 

7 

*Publisher  indicates  six  issues  are  published  yearly,  however  issues  1-2  are  combined  as  are  3-4  resulting 
in  only  4 publications  a year 


Simply  knowing  the  numbers  of  articles  per  publication  is  not  enough  to  select  a journal  for  publication.  It 
is  also  important  to  know  what  types  of  articles  are  generally  published.  Four  general  categories  (Descriptive, 
research,  editorial,  and  book  review)  were  used  to  review  all  five  journals.  Descriptive  articles  had  the  greatest 
diversity  and  included  description  of  theory,  practice,  models,  curriculum,  etc.  Research  encompassed  articles 
reporting  empirical  research  (both  quantitative  and  qualitative)  and  evaluation.  Introductions  and  conclusions  to 
special  issues  were  included  in  editorial  articles. 

Our  review  indicated  that  descriptive  articles  were  in  the  majority  and  articles  reporting  results  of  research 
were  second  (Table  2).  Three  of  the  top  five  appear  to  have  a good  balance  between  research  and  description  though 
all  three  publish  more  descriptive  articles  than  research  articles.  Educational  Technology  focuses  almost  exclusively 
on  descriptive  articles  and  the  Journal  of  Educational  Computing  Research  leans  heavily  toward  publishing  more 
research  articles. 


Table  2:  Summary  of  Types  of  Articles 


Journal 

Descriptive 

Research 

Editorial 

Book 

Review 

Educational  Technology  Research  and  Development 

40 

27 

7 

4 

Performance  Improvement  Quarterly 

43 

22 

8 

2 

Educational  Technology 

109 

9 

1 

0 

Journal  of  Educational  Computing  Research 

15 

63 

1 

13 

Instructional  Science 

33 

20 

6 

0 

The  third  factor  we  looked  at  when  reviewing  the  top  five  publications  related  to  topics.  A constant 
comparative  method  was  used  to  identify  themes  of  article  topics.  The  four  emergent  themes  were  design  issues, 
environment  issues,  student  attributes,  and  teacher  concerns  and  attributes.  The  following  section  presents  a 
summary  of  article  topics  organized  by  journal.  An  overview  of  each  journal  presenting  mission  and  goals,  and 
subscription  information  is  also  included. 

Educational  Technology  Research  and  Development  (ETR&D) 

ETR&D  is  a scholarly  journal  published  quarterly  be  the  Association  for  Educational  Communication  and 
Technology  (AECT).  Each  issue  includes  two  sections:  one  devoted  to  research,  both  practical  and  applied,  and  one 
to  design  and  development  issues.  Issues  frequently  include  book  and  international  reviews  as  well  as  research 
abstracts  (http://www.aect.org/Pubs/etrdweb/etr  d.html,  July  12.  2000).  Cabell's  Directory  of  Publishing 
Opportunities  in  Education,  5th  Edition  (19981  indicated  ETR&D  had  a circulation  of  close  to  5,000  and  an 
acceptance  rate  of  1 1-20%. 

Volumes  46  and  47  of  ETR&D,  a total  of  eight  issues,  were  reviewed  for  this  study.  The  large  number  in 
the  design  category  are  representative  of  the  development  section  of  the  journal.  Several  sub-theories  of  design 
emerged  including: 

• Theory  and  models  (systems  theory,  ARCS  model,  constructivism,  activity  theory); 

• Media  and  attributes  (authoring  systems,  multimedia,  ILS,  digital  manipulatives); 

• Design  principles  (needs  assessment,  evaluation);  and 

• Learning  and  cognition  (knowledge  management,  user-design,  cases,  cognitive  approach,  design  experts). 
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The  primary  topic  under  environment  issues  was  interaction.  Interaction  related  topics  included  learning 
communities,  cooperation,  and  electronic  classrooms.  Other  environment  topics  were  accessibility,  support  and 
organizational  complexity.  Student  attribute  topics  included  achievement,  recall  and  transfer,  preferences,  and 
reflection.  Topics  related  to  teacher  focused  heavily  on  teacher  preparation  but  also  looked  at  planning,  methods, 
management,  and  support  issues. 

Because  of  ETR&D’s  focus  on  research,  the  prime  category  that  did  not  seem  to  fit  any  of  the  themes  were 
topics  related  to  research  issues.  These  topics  related  to  changing  paradigms  and  theories,  and  research 
methodologies. 

Performance  Improvement  Quarterly  (PIQ) 

Bob  Mager  on  the  PIQ  website  (http://www.fsu.edu/~lsi/piaweb/*  July  1 2,  2000)  provides  insight  into  the 
focus  of  PIQ  with  the  following  quote: 

Serious  researchers  in  various  comers  of  performance  technology  will  find  PIQ  a scholarly  source 
of  current  research  by  some  of  their  more  notable  colleagues.  Practitioners  will  find  thought- 
provoking  reports  about  leading-edge  ideas  and  techniques  that  may  influence  their  practice.  If 
you're  serious  about  performance  technology,  you  should  be  reading  Performance  Improvement 
Quarterly. 

PIQ  is  a quarterly  publication  of  the  Learning  Systems  Institute  of  Florida  State  University  in  cooperation 
with  the  International  Society  of  Performance  Improvement. 

Volumes  11  and  12,  numbers  1-4  of  Performance  Improvement  Quarterly , a total  of  eight  issues  were 
reviewed  for  this  study.  There  were  three  special  issues  during  this  time:  global  distance  learning,  diversity,  and 
action  learning.  These  topics  also  appeared  in  the  regular  issues. 

It  was  more  difficult  to  fit  the  topics  in  Performance  Improvement  Quarterly  into  themes.  The  focus  of 
these  articles  was  on  the  corporate/business  environment.  Articles  related  to  instructional  design  were  featured  in 
almost  every  issue.  A variety  of  ID  models  were  described  and  research  results  related  to  emerging  models  of  design 
were  presented.  Environment  related  issues  also  appeared  prominently.  The  distance  learning  environment  was 
featured  frequently  and  included  topics  related  to  international  distance  learning,  delivery  systems,  history  and 
definitions.  Student  issues  included  performance/achievement,  transfer,  perceptions,  and  lifelong  learning.  Little 
coverage  was  given  to  teacher  topics.  Cost  benefit  and  analysis  appeared  frequently  as  did  topics  related  to  diversity. 

Educational  Technology 

Educational  Technology  is  a bi-monthly  publication  with  a readership  of  3000-4000.  Manuscripts  are 
reviewed  in-house  with  an  acceptance  rate  of  21-30%  (Cabells,  1999).  This  journal  does  not  focus  on  research 
articles  but  has  a more  general  slant.  “The  editors  are  looking  generally  for  articles  which  interpret  research  and/or 
practical  applications  of  scientific  knowledge  in  education  and  training  environments’’ 

(http://www.lbcovote.com/resource/pub/edutech.html),  A big  plus  with  this  journal  for  authors  is  the  turn  around 
time.  Notification  of  acceptance  is  usually  only  a couple  of  weeks  versus  the  months  of  waiting  that  accompany 
submission  to  other  journals. 

Volumes  38  and  39  were  reviewed  for  this  study.  One  special  issue  ‘integrating  cognition  and  affective 
domains  of  learning”  was  identified;  and  two  special  sections:  “Return  on  investment  in  educational  technology” 
and  “Intelligence  tutoring  and  learning  environments”.  The  articles  included  in  Educational  Technology  have  a more 
conversational  tone  than  the  articles  found  in  the  other  journals  reviewed  for  this  study.  Articles  also  tended  to  be 
shorter  and  included  a greater  variety  of  topics  and  viewpoints. 

Design  and  environment  themes  tended  to  dominate.  Design  sub-categories  were  similar  to  those  found  in 
ETR&D,  however  the  topics  included  in  the  sub-categories  were  more  varied.  Examples  include: 

• Theory  and  models  (systems  model,  social  systems,  interactive  model,  conversational  paradigm,  alternative 
paradigms,  postmodernism) 

• Media  and  attributes  (PowerPoint,  multimedia,  simulations,  expert  systems,  ILS,  games,  WWW,  screen  design) 

• Design  principles  (usability,  feedback,  planning);  and 

• Learning  and  cognition  (independent  learners,  affective  domain,  discovery  learning). 

Environment  interaction  issues  focused  on  the  virtual  classroom,  virtual  events,  online  education, 
community  of  practice,  and  international  collaboration.  Other  environment  topics  included  educational  reform  and 
specific  environments  of  distance  learning  and  Internet. 

Student  themes  fell  into  two  categories:  learning  and  achievement  including  future  skills  and  information 
literacy  skills;  and  characteristics  such  as  gender  differences,  mental  models,  and  self-regulation.  Few  article  topics 
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seemed  to  address  teacher  issues  or  characteristics.  Other  topics  included  in  Educational  Technology  were  systemic 
change,  diversity,  federal  policy,  and  future  visions. 

Journal  of  Educational  Computing  Research 

Baywood  Publishing  publishes  this  journal  eight  times  a year.  Articles  include  both  practice  and  theory 
with  an  emphasis  in  four  broad  areas:  1)  outcome  effects  of  educational  computing  applications,  2)  design  and 
development,  3)  research,  and  4)  foundations  in  educational  technology  (both  theory  and  historical). 

The  Journal’s  editors  view  the  term  “education”  in  its  broadest  sense.  The  use  of  computer-based 

technologies  at  all  levels  of  the  formal  education  system,  business  and  industry,  home-schooling,  lifelong 

learning  and  unintentional  learning  environments,  are  examined. 

(http://www.baywood.com/site/new2/viewbook.cfm?id=100133&c=,  July  12,  2000) 

According  to  Cabell’s  (1998),  the  Journal  of  Educational  Computing  Research  has  a readership  of  approximately 
1000  with  an  acceptance  rate  of  1 1-20%. 

The  primary  theme  that  emerged  from  a review  of  the  topics  in  volumes  18-21  of  Journal  of  Educational 
Computing  Research  was  student  attributes.  Three  sub-categories  were  identified: 

• Student  learning  (achievement,  prerequisite  skills,  critical  thinking  and  problem  solving); 

• Learning  and  cognitive  styles;  and 

• Characteristics  (attitudes,  perceptions,  anxiety,  self-efficacy,  gender  differences). 

While  student  themes  dominated,  the  other  three  themes  were  well  represented.  Design  topics  focused  on 
theory  and  models  and  media  attributes.  Interaction  and  cooperation  were  the  leading  topics  related  to  environmental 
issues.  Teacher  attribute  topics  included  perseverance/confidence,  learning  styles,  and  beliefs. 

Additional  topics  included  culture  and  social/ethics  issues.  Both  culture  from  a technology  environment 
viewpoint  and  an  international  viewpoint  appeared  in  many  issues.  Social/ethic  issues  included  equity,  access  and 
copyright  issues. 

Instructional  Science 

Unlike  the  previous  journals,  the  focus  of  this  journal  is  not  on  technology  but  on  learning.  Instructional 
Science  is  an  international  journal  of  learning  and  cognition  published  six  times  a year  by  Kluwer  Academic 
Publishers.  “The  primary  aim  of  Instructional  Science  is  to  promote  a deeper  understanding  of  the  nature,  theory  and 
practice  of  the  instructional  process  and  of  the  learning  to  which  it  gives  rise.” 
(http://www.wkap.nl/LOumalhome.htm/002Q-4277,  July  12,  2000). 

We  reviewed  eight  issues  of  Instructional  Science , Volume26  Number  1-6  and  Volume  27  Number  1-6. 
The  numbering  system  used  is  misleading.  Number  1 and  2 were  included  in  one  publication  as  were  numbers  3 and 
4.  In  the  two-year  time  period  there  were  four  special  issues:  generic  tutoring  environment  (GTE),  metacognition, 
use-system  interaction,  and  didactics.  Each  special  issue  included  at  least  one  editorial  as  an  introduction  to  the 
special  issue  and  some  also  included  concluding  comments. 

Topics  reviewed  student  issues/attributes  and  design  issues.  Student  topics  included: 

* Achievement  including  performance  and  vicarious  learning; 

* Characteristics  (self-regulation,  motivation,  reactions,  attitudes,  cognitive  styles);  and 

* Processing  (metacognition,  knowledge  construction,  cognitive  processing). 

Tools  to  assist  in  the  design  process,  such  as  authoring  tools  and  newly  developed  design 
software,  were  included  in  design  issues.  Also  included  in  this  theme  were  specific  media  and 
media  attributes  such  as  user-interface.  Environment  issues  and  teacher  issues  received  limited 
attention. 

Summary  of  Journals 

ETR&D  received  overwhelming  support  as  the  leading  journal  in  the  field  of  educational  technology.  The 
average  number  of  articles  included  in  each  issue  was  8.5.  The  types  of  articles  and  topics  covered  varied  greatly 
from  journal  to  journal  and  appear  to  be  dependent  on  both  the  mission  of  the  journal  and  the  intended  audience. 
Authors  should  carefully  review  journals  to  identify  those  that  most  closely  match  the  content  of  their  article  and 
the  audience  they  are  trying  to  reach.  Journals  should  also  be  reviewed  to  identify  the  dominant  writing  style  and 
take  that  into  consideration  when  selecting  a journal  for  publication. 

Advice  for  Authors 

The  second  part  of  the  email  survey  focused  on  the  process  of  writing  for  publication.  A review  of  previous 
articles  on  writing  for  publication  in  the  field  of  educational  technology  seemed  to  focus  on  identifying  and  targeting 
a journal  and  the  submission  process.  Newren  (1992)  contacted  editors  and  summarized  the  many  causes  for 
rejection.  Both  Clemente,  et.al.  (1990)  and  Thompson  (1995)  emphasized  the  need  to  concentrate  on  writing  style, 
targeting  a journal  and  following  submission  guidelines. 
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This  survey  was  interested  in  more  personal  suggestions.  Frequently  the  most  difficult  part  of  writing  is  the 
process.  While  not  all  individuals  responded  to  the  call  for  suggestions  for  those  beginning  a publication  career,  the 
suggestions  that  were  received  focused  more  on  the  process  of  writing  rather  than  the  submission.  Twenty 
individuals  responded  to  “What  one  suggestion  would  you  give  a doctoral  student  or  faculty  member  just  beginning 
his  or  her  writing  for  publication  career?” 

Three  themes  seem  to  emerge  with  analyses  of  the  responses:  work  with  a mentor,  believe  in  yourself  and 
what  your  are  researching,  and  write  frequently.  Several  suggested  that  the  individual  just  getting  started  work  with 
a mentor  or  collaborates  with  a faculty  member. 

“Apprentice  under/collaborate  with  a faculty  member  who  has  established  an  understanding  of  the 
processes,  strategies  and  priorities  of  publishing  scholarly  research . " 

“Find  a mentor  willing  to  read  your  papers  and  provide  you  with  constructive  criticism  prior  to  submitting 
them  for  publication.  ” 

Others  felt  it  was  important  to  not  only  believe  in  what  you  are  researching  but  to  believe  in  yourself. 
“Believe  in  yourself  and  believe  that  your  ideas  and  your  information  are  good.  People  just  starting  out 
tend  to  think  that  everyone  else  in  doing  great  things  but  that  their  own  work  is  trivial.... that's  a shame 
because  so  many  people  are  doing  great,  interesting  things  but  they  never  share  it  with  anyone  because 
they  are  embarrassed  or  they  don ’t  think  it's  good.  ’’ 

Still  others  felt  that  it  was  important  to  concentrate  on  one  or  two  projects  at  a time  and  to  write  every  day. 
“Write  every  day.  Work  on  thins  slowly  and  steadily  over  time,  reserving  the  most  creative  and  productive 
time  of  day  for  writing  time.  “ 

Certainly  paying  attention  to  detail  and  selecting  an  appropriate  journal  for  submission  are  important. 
However,  there  is  more  of  a personal  side  to  the  writing  process  and  this  was  emphasized  by  those  responding. 
Attitude  and  persistence  are  important.  Making  time  for  writing,  not  simply  “finding”  time,  and  collaborating  with 
others  are  instrumental  in  becoming  a successful  writer.  Perhaps,  most  important  of  all  is  belief  in  yourself  and  what 
you  are  writing.  With  the  fast  changing  world  of  technology,  exciting  research  is  going  on  and  all  have  something 
important  to  contribute  to  the  discussion. 

Conclusion  and  Recommendations 

The  purpose  of  this  study  was  to  identify  and  analyze  the  leading  journals  in  the  field  of  educational 
technology.  While  the  top  five  journals  all  examine  technology,  they  represent  disparate  views  in  the  field  and 
address  a wide  audience.  The  journals  are  all  unique  in  their  goals  and  mission,  writing  styles,  and  intended 
audience.  Certainly,  anyone  in  the  field  of  educational  technology  would  benefit  from  reading  any  of  these  journals 
but  would  gain  a much  broader  and  more  balanced  sense  of  research,  theories,  and  visions  from  reading  several. 
Collectively  they  provide  a wealth  of  information  to  assist  the  researchers  in  identifying  problems  and  areas  of  study 
needed  to  make  a contribution  to  the  growth  of  the  field  of  educational  technology. 

In  some  cases,  the  topics  presented  in  the  articles  are  very  similar.  However,  an  article  on  distance 
education,  for  example,  would  need  to  be  written  quite  differently  to  submit  to  ETR&D  than  to  submit  to 
Educational  Technology.  The  general  summaries  presented  here  serve  only  to  provide  a concise  look  at  the  myriad 
of  possibilities  for  publication  in  the  field  of  educational  technology.  It  is  imperative  that  those  preparing 
manuscripts  actually  review  the  journal  they  have  selected  for  submission 

It  should  also  be  noted  that  research  articles  do  not  constitute  the  majority  of  published  articles.  Frequently, 
novice  writers  make  the  false  assumption  that  only  results  of  empirical  research  will  be  published.  There  are  many 
opportunities  for  publication  of  articles  describing  projects,  special  instructional  methods,  new  models  and 
paradigms,  and  future  visions.  Topics  are  only  limited  by  the  experiences,  imagination,  creativity  and  interest  of  the 
writer. 

Practice  in  writing,  as  with  all  things  worth  doing,  is  important.  The  experts  represented  in  this  study  as 
well  as  colleagues  we  have  visited  with  indicate  the  importance  of  writing  on  a regular  basis.  Frequently,  in  the  day 
to  day  workday,  other  activities  take  precedence  over  writing.  Time  should  be  blocked  out  for  writing  as  a scheduled 
activity  and  care  should  be  taken  to  guard  and  use  that  time  to  polish  the  skills  needed  for  successful  writing. 

In  summary,  while  the  response  rate  to  the  email  survey  was  lower  than  hoped  for,  the  responses  did 
represent  what  practitioners  in  the  field  of  educational  technology  are  reading  and  which  journals  were  most 
respected  by  a broad  group  of  leading  professionals  in  the  field.  ETR&D  was  emphatically  identified  as  the  leading 
journal  in  the  field  of  educational  technology.  Broad  categories  of  topics  emerged  from  a review  of  the  top  five 
journals.  These  categories  (with  some  examples)  include: 
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• design  issues  (models,  theory,  principles,  media,  and  media  attributes); 

• student  issues/attributes  (achievement,  learning  styles,  attitudes,  perceptions); 

• environment  issues  (interaction,  collaboration,  international);  and 

• teacher  issues  (pre-service  teacher,  confidence,  attitudes,  methodology). 

Dissemination  of  research  in  educational  technology  is  the  responsibility  of  all  professionals  in  the  field. 
Publication  in  recognized  journals  contributes  to  the  growth  and  recognition  of  educational  technology.  The 
information  discussed  in  this  article  provides  a base  for  selecting  a journal  for  publication.  It  is  up  to  each  individual 
author  to  provide  the  personal  side  of  creativity,  dedication,  and  perseverance. 
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Abstract 

The  purpose  of  this  study  was  to  examine  current  attitudes  and  knowledge  of  pre-service  teachers  towards 
assistive  technology  and  to  develop , implement , and  evaluate  a mini-workshop  on  assistive  technology  to  better 
prepare  regular  classroom  teachers  for  the  inclusive  classroom.  A pre-post  survey  design  was  used.  Data  from  the 
pre-survey  provided  demographic  information  as  well  as  documenting  current  attitudes  and  knowledge.  Participants 
were  168  students  enrolled  in  a computer  applications  course  for  elementary  teachers.  A mini-workshop  on  assistive 
technology  was  presented  by  experts  in  the  field.  Data  was  analyzed  to  determine  impact  of  the  mini-workshop  on 
attitude  and  knowledge  and  to  detect  differences  based  on  completion  of  a diversity  course , having  a disabled 
family  member  and  having  a disabled  friend.  Results  indicated  a continued  need  to  include  assistive  technology 
under  the  broader  umbrella  of  technology  in  teacher  preparation  programs. 

Introduction 

During  the  past  decade,  the  demand  for  technology  literate  teachers  has  increased  dramatically.  Both  the 
government  and  the  public  support  the  need  for  excellence  and  equity  in  technology  integration,  though,  funding 
opportunities  to  support  technology  related  professional  development  have  focused  on  the  in-service  teacher. 
Technology  experiences  for  pre-service  teachers  have  centered  around  one  basic  class,  usually  computer  based,  and 
limited  modeling  by  a few  innovative  methods  instructors.  The  content  of  the  basic  course,  too  frequently,  focused 
heavily  on  computer  skills  and  minimally  on  integration  of  technology.  Little  if  any  reference  was  made  to  assistive 
technology  and  appropriate  application  in  the  regular  classroom. 

This  paper  presents  the  results  of  a preliminary  study  to  evaluate  the  use  of  a mini-workshop  on  assistive 
technology  presented  by  special  education  experts  intended  to  assist  pre-service  teachers  in  developing  an 
awareness  of  the  variety  of  assistive  technologies  available  and  the  teacher’s  role  in  using  these  devices  or 
equipment  in  the  regular  classroom. 

Background  and  Purpose 

Mention  the  word  technology  to  someone  today  and  the  first  thing  that  comes  to  mind  is  computers  and  the 
World  Wide  Web.  However,  when  it  comes  to  education,  that  is  too  narrow  of  definition.  In  teacher  preparation 
programs,  students  used  to  be  required  to  take  a media  class  that  addressed  a variety  of  technologies  from  16mm 
films  to  overhead  projectors  to  computers.  Because  of  the  technological  advances  in  the  past  ten  years,  these  classes 
have  evolved  to  where  the  primary  focus  is  currently  on  computer  use  and  curriculum  integration.  Students  have 
little  exposure  to  other  effective  and  appropriate  technologies  that  can  be  used  in  the  regular  classroom  to  promote 
and  enhance  teaching  and  learning.  One  major  category  of  technologies  has  been  virtually  ignored,  assistive 
technology. 

First,  it  would  seem  appropriate  to  provide  a definition  of  assistive  technology  or  AT.  The  following 
definition  appeared  in  the  Technology-related  Assistance  for  Individuals  with  Disabilities  Act  of  1988  or  the  Tech 
Act  (P.L.  100-407)  and  has  been  adopted  in  the  Individuals  with  Disabilities  Education  Act  (IDEA,  1990).  Assistive 
technology  is  “any  item,  piece  of  equipment  or  product  system,  whether  acquired  commercially  off  the  shelf, 
modified,  or  customized,  that  is  used  to  increase,  maintain,  or  improve  functional  capabilities  of  individuals  with 
disabilities.”  [20  U.S.C.  Chapter  33,  Section  1401  (25)].  As  you  can  see,  this  definition  is  broad  and  can  encompass 
a range  of  devices  from  low  technology  to  high  technology  items  as  well  as  software.  It  certainly  expands  the 
interpretation  of  technology  beyond  the  computer  to  include  even  simple  tools  that  can  be  used  to  enhance  learning 
for  all  individuals. 

The  1997  reauthorization  of  the  Individuals  with  Disabilities  Education  Act  (IDEA)  mandated  that  students 
with  disabilities  learn  and  be  evaluated  with  their  peers  (Goldberg,  1999;  Derer,  Lewis,  & Rieth,  1996;  Lewis, 
1998).  With  this  mandate,  more  school  districts  are  implementing  inclusion  of  students  with  disabilities  into  the 
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regular  classroom.  This  in  turn  has  created  greater  need  for  awareness  of  instructional  methods  and  assistive  devices 
to  help  these  students  become  a productive  member  of  the  classroom  with  a minimum  amount  of  disruption. 

Previous  research  in  the  area  of  assistive  technology  and  teacher  preparation  appears  to  be  very  limited 
(Derer,  Polsgrove  & Rieth,  1996).  There  was  discussion  of  the  needs  in  this  area  and  some  suggestions  as  to  how  to 
address  this  need,  however,  very  few  of  the  articles  reflected  an  actual  implementation  of  in-service  and  pre-service 
training  in  assistive  technology  for  administrators,  teacher  and  other  school  personnel.  For  example  Bryant,  Erin, 
Lock,  Allan  and  Resta  (1998)  discussed  the  need  for  higher  education  faculty  responsible  for  special  education 
teacher  preparation  programs  to  explore  ways  to  better  prepare  teachers  to  work  with  students  using  assistive 
technology  devices  but  did  not  address  this  same  need  for  “regular”  classroom  teachers. 

In  the  inclusive  classroom,  assistive  technology  can  provide  support  for  both  students  and  teachers  (Mebler,  Hadaian 
& Ulman,  1999).  However,  in  order  for  this  to  happen  both  the  student  and  the  teacher  must  understand  and  know 
how  to  use  the  devices.  Planning  and  policy  at  all  levels,  (classroom,  school  and  district)  are  essential  to  achieve 
this. 

It  appears  that  regular  classroom  teachers  are  not  currently  receiving  knowledge  and  skills  related  to 
assistive  technology.  One  way  to  address  this  need  is  at  the  pre-service  or  teacher  preparation  level.  The 
International  Society  for  Technology  in  Education  (ISTE)  is  the  professional  education  organization  responsible  for 
recommending  guidelines  for  accreditation  to  National  Council  for  Accreditation  of  Teacher  Education  (NCATE) 
for  programs  in  educational  computing  and  technology  teacher  preparation.  Included  in  the  recommended 
foundations  in  technology  for  all  teachers  are  two  standards  that  point  to  the  need  for  assistive  technologies  to  be 
included  in  the  teacher  preparation  program.  First,  beginning  teachers  should  be  able  to  “plan  and  participate  in 
activities  that  encourage  lifelong  learning  and  promote  equitable,  ethical,  and  legal  use  of  computer/technology 
resources”  (ISTE,  I.B).  In  addition,  they  should  be  able  to  “demonstrate  awareness  of  resources  for  adaptive 
assistive  devices  for  student  with  special  needs”  (ISTE,  I.B. 5). 

The  ISTE  foundation  standards  guide  the  activities  and  instruction  in  the  Applications  of  Technology  in 
Elementary  Education  Course.  This  is  an  undergraduate  teacher  preparation  class  that  engages  the  student  in  the  use 
of  technology  as  an  educational  tool.  Opportunity  is  provided  for  students  to  explore  and  utilize  technology 
applications  that  enhance  the  teaching  and  learning  process.  Emphasis  is  placed  on  the  design,  development  and 
delivery  of  effective  communication  and  learning  activities.  Until  fall,  1999,  the  standard  related  to  assistive 
technology  was  not  addressed.  This  study  looks  at  an  effort  to  assist  pre-service  teachers  in  achieving  an  awareness 
of  assistive  technology,  the  appropriate  application  of  AT  in  the  regular  classroom,  and  the  role  of  the  classroom 
teacher  in  using  assistive  technology  to  enhance  teaching  and  learning  for  all  students. 

Purpose 

Very  little  supporting  research  for  the  incorporation  of  assistive  technologies  into  the  teacher  preparation 
program  for  regular  classroom  teachers  was  found  in  a review  of  the  literature.  Therefore,  a decision  was  made  to 
conduct  a preliminary  study  to  determine  the  students’  knowledge  of  and  attitudes  toward  assistive  technology. 

The  purpose  of  this  study  was  three-fold.  First,  researchers  were  interested  in  identifying  pre-service 
teachers  current  knowledge  of  and  attitude  toward  assistive  technology.  If  the  ISTE  standard  related  to  assistive 
technology  is  to  be  achieved,  than  a knowledge  of  current  status  related  to  this  standard  must  be  assessed  to  establish 
baseline  data.  Second,  it  was  important  to  assess  the  impact  of  a mini-workshop  on  both  knowledge  and  attitudes. 
Similar  to  the  majority  of  pre-service  method  classes,  time  restraints  and  current  curricula  needs  in  the  Computer 
Application  class  restricted  the  amount  of  time  that  could  be  allotted  to  this  topic.  There  was  a need  to  discover  if 
the  mini-workshop  approach  would  be  an  effective  means  to  assist  students  in  achieving  the  minimum  standard 
related  to  assistive  technology.  Last,  instruments  and  procedures  needed  to  be  evaluated  for  their  potential  use  in 
future  studies.  The  inclusion  of  assistive  technology  into  the  pre-service  teacher  education  program  was  a much- 
neglected  area  of  study.  Both  instruments  and  procedures  that  are  valid  and  reliable  needed  to  be  developed  to 
further  this  field  of  study. 

Methodology 

A pre-post  test  design  was  used  for  this  study.  The  class  instructor  conducted  survey  administration.  At  the 
beginning  of  the  class  period,  participants  were  asked  to  complete  the  pre-survey.  After  surveys  were  collected,  two 
experts  in  the  field  conducted  a mini-workshop  on  assistive  technology.  A diverse  sampling  of  assistive  technologies 
were  presented  and  discussed.  Students  had  the  opportunity  to  interact  with  the  technologies  shown  and  to  ask 
questions  of  the  two  experts.  The  entire  time  allotment  for  both  the  pre-survey  data  collection  and  the  mini- 
workshop was  approximately  80  minutes.  Within  two  weeks  during  another  class  period,  students  were  asked  to 
complete  the  post-survey. 
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Mini-workshop 

The  mini-workshop  began  with  a review  of  the  definitions  of  assistive  technology  and  the  Tech  Act. 
Various  low  and  high  tech  devices  were  shown,  demonstrated  when  applicable,  and  applications  for  specific 
disabilities  were  identified.  The  students  then  had  the  opportunity  to  handle  and  explore  the  various  devices.  Other 
options  for  assistive  technology  were  briefly  reviewed.  Emphasis  was  placed  on  the  role  and  responsibility  of 
classroom  teachers  to  make  their  classroom  accessible.  Assistive  technology  can  provide  the  means  for  accessibility 
for  some  children  with  disabilities.  Students  had  the  opportunity  to  ask  questions  throughout  the  workshop  and  were 
encouraged  to  talk  with  the  presenters  individually  and  examine  the  technology  in  more  detail  at  the  conclusion  of 
the  workshop. 

Instrument 

The  instrument  used  for  this  study  was  adapted  from  an  instrument  with  known  validity  and  reliability 
previously  used  by  one  of  the  researchers.  While  the  original  instrument  was  not  appropriate  for  use  in  these 
circumstances,  it  did  provide  guidance  in  the  selection  of  items  to  include  that  addressed  attitude  and  knowledge 
related  to  assistive  technology.  Using  this  instrument  as  a guide,  a special  education  professional  and  an  educational 
technology  professional  worked  together  to  develop  a 20-question  survey  for  this  study.  A total  of  20  questions  were 
designed  to  measure  students’  knowledge  of  and  attitudes  toward  assistive  technology  and  students  with  disabilities. 
A six-item  Likert  scale  from  strongly  disagree  to  strongly  agree  was  used  as  a response  set.  In  addition  to  the  20 
Likert  scale  items,  the  survey  included  demographic  items  to  describe  the  population.  Three  additional  variables 
were  also  investigated.  They  were: 

1 . Completion  of  a required  course  on  diversity, 

2.  Having  a family  member  who  is  disabled, 

3.  Having  a close  friend  who  is  disabled. 

Both  professionals  reviewed  the  instrument  and  both  contributed  to  revisions  to  insure  content  validity. 
Once  data  was  collected,  a reliability  analysis  was  conducted.  Using  all  20  items  on  the  survey,  the  instrument  had  a 
reliability  coefficient  of  .71  using  the  Guttman  split-half.  To  somewhat  control  for  the  short  length  of  the  instrument 
reliability  was  recomputed  using  a split-half  corrected  by  Spearman-Brown  prophecy  formula  resulting  in  a 
reliability  coefficient  of  .72.  This  was  still  in  the  low  range  but  of  some  value  as  this  was  a preliminary  study  and  the 
instrument  was  only  being  used  for  group  measurement  and  not  individual  measurement.  Reliability  results 
indicated  that  before  further  data  is  collected  revisions  should  be  made  in  the  survey  instrument  to  improve 
reliability. 

Participants 

A convenience  sample  was  used  for  this  preliminary  study.  Participants  in  this  study  were  the  students 
enrolled  in  a Computers  Application  for  Elementary  Teachers  class  during  the  1999-2000  school  year.  There  were 
five  sections  of  this  class  each  of  the  two  semesters.  A pre-post  design  was  used.  One  hundred  sixty  eight  students 
completed  the  pre-survey;  1 54  students  responded  to  the  post-survey. 

Information  collected  on  the  pre-survey  was  used  to  describe  the  participants.  The  majority  of  them  were 
female(85%),  juniors  (78%),  and  20-21(72%)  years  of  age  with  an  age  range  from  19  to  44  years.  Almost  three- 
quarters  (73.1%)  had  completed  the  diversity  class.  About  one-fourth  had  a family  member  who  was  disabled 
(24%).  Similarly,  approximately  one-fourth  had  a friend  who  was  disabled  (23%). 

Results 

Data  were  analyzed  three  ways.  First  pre-survey  frequencies  were  reviewed  to  identify  current  attitude  and 
knowledge  and  to  identify  areas  of  concern.  Second,  post-survey  responses  were  analyzed  to  see  if  the  same  areas  of 
concern  held  true  after  participants  completed  the  mini-workshop.  Finally  comparisons  in  mean  scores  were  used  to 
detect  differences  between  pre  and  post  as  well  as  differences  in  current  attitudes  and  knowledge  based  on 
independent  variables:  diversity  class,  disabled  friend,  and  disabled  family  member, 

Pre-Survey 

Pre-survey  data  was  used  to  measure  current  knowledge  of  and  attitudes  toward  assistive  technology. 
Starred  items  reflect  negatively  worded  items  or  items  where  a negative  response  was  desirable.  For  purposes  of 
analysis  and  to  simplify  comparisons,  all  negatively  worded  items  were  recoded  to  reflect  agreement  level. 
However,  actual  frequencies  were  presented  in  all  tables,  A 25%  level  of  disagree  was  set  by  the  researcher  to 
identify  areas  of  concern. 

There  were  ten  items  included  to  measure  attitudes  toward  assistive  technology  and  special  needs  students. 
Out  of  the  ten  attitude  items,  five  were  above  a 75%  agreement  level.  All  children  have  the  same  need  for  praise  and 
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disabled  children  strive  as  hard  as  others  received  the  highest  agreement  ratings  of  97%  agree  a little  to  strongly 
agree. 

Five  areas  of  concern  were  identified.  The  statement  “disabled  children  are  more  self-confident  than  other 
children”  received  the  lowest  agreement  rating.  Eighty-nine  percent  indicated  that  they  disagreed  a little  to  strongly 
disagreed  with  this  statement.  Seventy-one  percent  disagreed  a little  to  strongly  disagreed  with  the  statement  “there 
should  not  be  special  schools  for  disabled  children”.  Additionally,  47%  disagreed  a little  to  strongly  disagreed  that 
disabled  children  should  compete  with  others;  36%  disagreed  a little  to  strongly  disagreed  that  children  will  not  be 
uncomfortable  with  disabled  children;  and  33%  disagreed  a little  to  strongly  disagreed  with  the  statement  “most 
disabled  children  do  not  feel  sorry  for  themselves”. 

Responses  to  the  ten  knowledge  questions  were  generally  positive.  Nine  of  the  knowledge  items  received 
agreement  ratings  of  over  92%  agreed  a little  to  strongly  agreed.  Ninety-nine  percent  agreed  a little  to  strongly 
agreed  that  assistive  technologies  could  enhance  the  learning  of  disabled  children.  No  response  was  low  enough  to 
identified  an  area  of  concern.  However,  the  lowest  agreement  (76%  agreed  a little  to  strongly  agreed)  was  with  the 
statement  “assistive  technologies  are  (not)  all  high-tech”. 

Post-survey  Responses 

Similar  to  the  pre-survey  responses,  five  attitude  items  received  agreement  ratings  of  over  80%  agreed 
slightly  to  strongly  agreed.  Rank  order  of  items  based  on  agreement  ratings  were  almost  identical  when  comparing 
post-survey  frequencies  to  pre-survey  frequencies.  However,  most  responses  received  higher  ratings.  Significant 
differences  will  be  discussed  in  the  next  section,  comparisons. 

All  ten  knowledge  items  received  agreement  ratings  of  over  89%  agreed  slightly  to  strongly  agreed  on  the 
post-survey.  Rank  order  items  were  very  similar  to  those  from  the  pre-survey.  The  most  notable  change  was  from 
76%  agreed  slightly  to  strongly  agreed  on  the  pre-survey  to  90%  on  the  post-survey  that  assistive  technologies  are 
not  al  high-tech. 

Comparisons 

Overall  attitude  and  knowledge  scores  for  both  pre  and  post-surveys  were  computed.  Negative  items  were 
recoded  and  then  a sum  of  responses  to  the  ten  items  in  each  category  was  computed.  This  provided  an  overall 
attitude  score  and  an  overall  knowledge  score  for  comparison  purposes.  An  alpha  level  of  <.05  was  used  to 
determine  significance. 

The  pre-survey  attitude  and  knowledge  scores  were  compared  to  detect  any  differences  in  mean  scores.  T- 
values  were  computed  to  identify  any  statistically  significant  differences.  No  change  was  detected  in  attitude  scores, 
but  results  indicated  significantly  higher  post-survey  knowledge  scores  (t=4.857,  p<.001). 


Table  1:  Change  in  Attitude  and  Knowledge,  Comparison  of  Pre-survey  and  Post- survey  overall 
scores . 


N 

Mean 

SD 

t-value 

Significance 

Attitude 

Pre 

168 

41.80 

4.41 

1.047 

.296 

Post 

154 

42.35 

4.93 

Knowledge* 

Pre 

168 

49.91 

4.63 

4.857 

<.001 

Post 

154 

52.56 

5.15 

* Significance  <.05 


Data  from  the  pre-survey  was  used  to  examine  overall  attitude  and  knowledge  based  on  the  three 
independent  variables  diversity  class  (Table  2),  disabled  family  member  (Table  3),  and  disabled  friend  (Table  4).  It 
was  hypothesized  that  those  students  who  had  completed  the  diversity  class  and  those  students  who  had  either  a 
disabled  friend  or  family  member  would  have  a higher  overall  score  on  attitude  and  knowledge.  Thus,  a one-tailed 
test  of  significance  was  used. 

Students  who  had  completed  the  class  on  diversity  had  a higher  overall  attitude  score,  but  a lower  overall 
knowledge  score.  However,  no  statistically  significant  differences  were  detected  based  on  this  sample. 
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Table  2:  Comparison  Of  Current  Attitude  And  Knowledge  Based  On  Completion  Of  Diversity  Class 


N 

Mean 

SD 

t-value 

Significance 

Attitude 

Yes 

122 

42.01 

4.33 

.904 

.184 

No 

45 

41.31 

4.66 

Knowledge 

Yes 

122 

49.67 

4.74 

1.264 

.104 

No 

45 

50.69 

4.22 

* Significance  <.05 

Attitudes  of  those  with  a disabled  family  member  did  not  appear  to  be  much  different  than  attitudes  of 

those  without  a disabled  family  member.  As  expected,  overall  knowledge  scores 

were  higher  for  those  with  a 

disabled  family  member.  However,  on  neither  attitude  nor  knowledge 

were  the  differences  statistically  significant. 

Table  3:  Comparison  Of  Current  Attitude  And  Knowledge  Based  On  Having  a Disabled  Family 

Member 

N 

Mean 

SD 

t-value 

Significance 

Attitude 

Yes 

40 

41.28 

3.88 

.868 

.194 

No 

128 

41.97 

4.56 

Knowledge 

Yes 

40 

50.85 

4.84 

1.476 

.071 

No 

128 

49.62 

4.54 

* Significance  <.05 

On  both  attitude  and  knowledge,  those  having 

a disabled  friend  scored  higher.  A statistically  significant 

difference  was  detected  on  attitude  based  on  having  a disabled  friend. 

Table  4:  Comp 

arison 

Of  Current  Attitude  And  Knowledge  Based  On  Having  a Disabled  Friend 

N 

Mean 

SD 

t-value 

Significance 

Attitude* 

Yes 

39 

43.03 

4.48 

1.911 

.029 

No 

128 

41.53 

4.21 

Knowledge 

Yes 

39 

50.21 

5.02 

.471 

.319 

No 

128 

49.80 

4.54 

* Significance  <.05 


Discussion 

This  study  attempted  to  look  at  three  areas:  current  attitude  and  knowledge,  impact  of  mini-workshop,  and 
assessment  instruments  and  procedures.  Discussion  will  be  organized  around  these  three  themes  concluding  with  a 
discussion  of  future  research  plans  in  this  area. 

Current  Attitude  and  Knowledge 

Data  from  pre-survey  responses  were  used  to  determine  current  pre-service  teachers*  attitudes  toward 
students  with  special  needs  and  knowledge  about  assistive  technology.  Attitudes  were  very  positive.  It  is  felt  that 
this  may  not  be  so  much  a factor  of  teacher  preparation  programs  but  relates  to  students’  personal  schooling 
experiences  and  general  social  attitudes.  This  could  also  be  a result  of  the  age  and  sex  of  participants.  Participants  in 
this  study  were  predominately  traditional  female  college  students  with  an  average  age  of  21.3.  If  there  had  been  a 
larger  population  of  older,  non-traditional  students  and  more  male  elementary  pre-service  teachers,  we  could  have 
examined  differences  in  attitudes  based  on  age  and  sex. 

While  recognizing  that  attitudes  are  difficult  to  change,  there  were,  however,  some  areas  of  concern  that 
need  to  be  addressed  in  the  teacher  preparation  program.  The  participants  in  this  study  will  be,  for  the  most  part. 
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traditional  classroom  teachers  and  not  special  education  teachers.  It  was  surprising  to  find  that  almost  three-fourths 
felt  that  there  should  be  special  schools  for  students.  It  is  difficult  to  determine  participants  reasoning  and 
perspectives  on  this  without  further  investigation. 

Perhaps  more  troubling  was  the  fact  that  over  one  third  felt  that  children  in  a traditional  classroom  would 
be  uncomfortable  with  a disabled  child  and  that  disabled  children  feel  sorry  for  themselves.  These  attitudes  still 
persist  even  with  today’s  strong  focus  on  acceptance  of  diversity. 

We  had  expected  that  students  who  had  already  taken  the  required  class  on  diversity  would  have  a more 
positive  attitude  toward  disabled  children  and  would  have  more  knowledge  of  IDEA  and  assistive  technology.  This 
did  not  prove  to  be  the  case  with  this  sample.  The  content  of  this  course  has  changed  with  the  emphasis  on  diversity 
and  perhaps  other  issues  have  replaced  the  previous  inclusion  of  topics  related  to  special  needs  populations. 

Analysis  of  the  fall  semester  data  had  indicated  differences  in  attitude  based  on  having  a disabled  family 
member.  This  difference  was  not  detected  with  the  full  sample.  In  both  semesters,  approximately  the  same 
percentage  of  participants  indicated  that  they  had  a disabled  family  member,  so  this  probably  did  not  account  for  the 
change.  Future  research  will  be  needed  in  this  area  including  identification  of  the  relationship  of  the  disabled  family 
member  to  explore  this  area  in  more  detail. 

Knowledge  responses  were  also  surprisingly  positive.  Students  appeared  to  have  at  least  an  awareness  of 
assistive  technology.  However,  it  must  be  cautioned  that  this  positive  response  could  be  misleading.  As  the  survey 
was  self-report,  students  may  have  been  marking  what  they  felt  was  the  expected  response  and  therefor  responses 
are  not  representative  of  actual  knowledge. 

Impact  of  Mini-workshop 

Because  of  time  restraints  and  an  already  overloaded  curriculum,  only  one  class  period  could  be  devoted  to 
the  presentation  of  assistive  technologies.  This  short,  one-time  exposure  was  not  enough  to  make  a difference  in 
attitudes.  This  was  expected  as  attitudes  tend  to  be  firmly  entrenched  and  difficult  to  change  even  with  multiple 
exposures. 

The  mini-workshop  was  effective  in  changing  knowledge.  However,  there  are  several  variables  at  work 
here.  It  is  not  possible  at  this  time  to  determine  if  this  change  was  a factor  of  the  technologies  displayed  or  of  the 
instructor.  Participants  were  exposed  to  and  had  the  opportunity  to  handle  a variety  of  “toys”.  It  is  possible  that  this 
actual  hands-on  approach  was  the  largest  contributor  to  the  changes  in  knowledge  detected.  A team  of  special 
education  instructors  presented  all  of  the  mini-workshops.  How  much  of  the  change  was  a factor  of  the 
specialization  and/or  personality  of  the  presenters  needs  to  be  examined  in  future  research. 

Instruments  and  Procedures 

While  the  instrument  did  show  to  be  valid  and  reliable,  changes  should  be  made  to  improve  both  reliability 
and  usability.  The  number  of  items  needs  to  be  increased  to  improve  reliability.  In  addition,  though  the  original 
survey  adapted  for  use  in  this  study  included  negatively  worded  items  it  is  felt  that  these  items  adversely  affected 
both  student  responses  and  reliability  of  the  instrument  and  should  be  revised  prior  to  future  use. 

Future  Directions 

This  preliminary  study  supports  the  need  for  the  inclusion  of  assistive  technology  in  the  preparation  of 
teachers.  Our  efforts  in  investigating  the  best  way  to  provide  this  content  will  continue. 

First,  the  instrument  will  be  revised  based  on  the  results  of  this  study.  Negatively  worded  items  will  be 
replaced  with  positive  statements.  Attitudes  will  continue  to  be  part  of  the  pre-survey  but  will  be  eliminated  from  the 
post  survey  as  it  is  difficult  to  impact  a change  on  attitudes.  The  number  of  knowledge  items  will  be  increased  and 
reworded  to  more  closely  reflect  the  content  of  the  mini-workshop.  The  format  of  the  knowledge  items  will  be 
revised  from  a Likert  response  set  to  a multiple  choice  format  that  supports  the  overall  expected  competencies 
related  to  assistive  technologies. 

Second,  the  workshop  be  varied.  A quasi-experimental  design  will  be  used  to  examine  factors  impacting 
change  on  knowledge.  Experimental  groups  will  be  used  to  examine  the  impact  of  both  the  hands-on  use  of  the 
technologies  and  the  instructors.  In  addition,  the  creation  of  software  for  self-study  of  assistive  technology  is  being 
explored. 

While  this  preliminary  study  was  not  without  limitations,  it  proved  useful  on  several  levels.  Both  individual 
course  planning  and  program  planning  will  benefit  from  the  results  of  this  study.  The  technology  for  teachers  class 
and  the  required  diversity  class  will  be  reviewed  for  content  and  the  addition  of  assistive  technology  components 
that  prepare  teachers  for  their  role  in  adapting  the  regular  classroom  for  special  needs  children.  Within  programs, 
emphasis  on  the  integration  of  technology,  including  assistive  technology,  across  the  teacher  preparation  program 
will  increase. 

This  study  was  successful  in  building  collaboration  between  special  education  and  technology  programs. 
This  sets  the  stage  for  planning  and  funding  of  technology  needs  within  the  college  and  established  a framework  for 
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further  collaborative  research  between  these  two  programs.  It  also  serves  as  a model  for  technology  collaboration 
with  other  programs,  departments,  and  colleges  strengthening  the  technology  integration  across  programs  that  is 
essential  for  the  preparation  of  technology  literate  teachers. 
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Abstract 

Educators  today  are  under  tremendous  pressure  to  make  use  of  the  latest  in  technology  while  continuing  to 
provide  quality  educational  experiences  for  students.  Quality  staff  development  is  essential  and  costly,  especially  in 
rural  areas.  This  study  looked  at  the  use  of  mentoring  as  a positive  strategy  for  building  teacher  confidence  in 
technology  application  skills.  The  study  emerged  as  a grant  to  direct  funds  into  rural  schools  and  was  specifically 
targeted  for  educational  improvement  through  technology  implementation.  Qualitative  and  quantitative  data 
indicated  that  mentoring  was  a positive  influence  on  teacher  technology  confidence,  technology  integration,  school 
climate,  and  staff  development  effectiveness. 

Technology  Manpower  Needs  and  Teacher  Technology  Training 

Career  opportunities  of  today  suggest  that  of  the  approximately  600,000  job  openings  currently  advertised 
450,000,  are  related  to  technology  and  knowledge  of  computer  applications.  Using  technology  as  a tool  to  improve 
teaching  and  learning  is  a critical  need  in  schools,  but  crucial  to  being  employed  in  the  future  is  knowing  how  to  use 
advanced  technology  in  a useful  way  (Dennis,  1998).  In  an  Office  of  Technology  Assessment  (OTA)  study  of  the 
use  of  technology  in  education,  lack  of  training  and  limited  knowledge  about  computers  were  the  most  commonly 
cited  reasons  for  non-use  of  computers  (1995).  Recent  studies  continue  to  report  that  properly  trained  teachers 
make  the  difference  between  success  or  failure  of  technology  integration  efforts  (Siegel,  1995).  For  technology 
based  learning  to  be  effective,  teachers  must  select  materials  that  help  meet  carefully  defined  instructional  objectives 
and  integrate  them  into  learning  experiences.  It  is  time  to  acknowledge  the  vital  role  that  teachers  play  in  the 
successful  use  of  technology  for  learning  with  support  (Mellon,  1999).  The  10-year  ACOT  project  (Apple 
Classrooms  of  Tomorrow,  Saltpeter,  1998)  recommended  that  30  percent  of  available  technology  resources  be 
dedicated  to  providing  ongoing  staff  development.  A Time  Magazine  report  claims  that  schools  spent  about  $88  this 
year  per  student  on  computer  equipment,  but  only  $6  per  student  on  computer  training  for  teachers.  Although  80 
percent  of  schools  have  Internet  access  only  20  percent  of  teachers  polled  in  this  survey  felt  prepared  to  use 
technology  in  their  classes  (Nellen,  1999).  A 1999  study  by  Market  Data  Retrieval  (MDR)  found  that  61  percent  of 
the  teachers  surveyed  felt  either  “not  at  all  prepared”  or  only  “some  what  prepared”  to  effectively  use  new 
technology  (Mckenzie,  2000 

Professional  Development 

Statham  and  Torell  (1996)  have  identified  professional  development  as  an  essential  condition  necessary  to 
maximize  student  achievement.  Teachers  are  the  keys  to  success,  and  training  the  teachers  is  essential.  They  state: 
“A  commitment  to  technology  integration  includes  a commitment  to  teacher  training.”  The  success  or  failure  of 
technology  is  more  dependent  on  human  and  contextual  factors  than  on  hardware  or  software  (Valdez,  et.al.  1999). 
Based  on  survey  data  Becker  was  able  to  determine  that  certain  variables  had  ‘important  independent  relationships 
to  teachers  use  and  valuation  outcomes’  (Becker,  1999).  Significant  factors  were:  quality  connectivity,  computer 
expertise,  teacher  pedagogical  beliefs  and  practices,  and  adequate  professional  development  (Becker,  1999). 
Research  indicates  that  most  school  districts  spend  less  than  a quarter  of  their  computer  budgets  on  training  (Bruder, 
1993).  Too  often  faculty  professional  development  features  one-shot  workshops  with  limited  support  and  follow-up 
for  integration  purposes  (Hargreaves  & Fullan,  1992).  Joyce  and  Showers  (1995)  have  argued  that  teacher 
development  should  be  innovation-related,  continuous  over  several  sessions  and  involve  a variety  of  formal  and 
informal  training  sessions  to  meet  the  needs  of  the  teacher.  Their  model  emphasizes  the  need  for  the 
leamer(teacher)  to  be  shown  how  the  application  works,  be  provided  an  opportunity  to  practice  with  the  application, 
then  receive  follow-up  support  to  allow  for  further  practice  and  related  critical  feedback 

Recently  staff  developers  spotlighted  critical  issues  facing  schools  and  identified  mentoring  programs  as 
useful  in  teacher  improvement  (Ganser,  2000).  Technology  changes  so  fast,  some  teachers  may  not  know  how  to 
keep  up  (Bray,  1999).  It  would  be  most  valuable  to  provide  both  time  and  opportunities  for  on  the  job  professional 
development.  On  activities  using  technology  applications  of  skill  in  the  work  world,  students  of  teachers  with  more 
that  ten  hours  of  training  significantly  outperformed  students  whose  teachers  had  five  or  fewer  hours  of  training 


(Becker,  1999).  The  use  of  on-sight  teams  or  mentor  groups  can  provide  valuable  guidance  and  increase  the 
effectiveness  of  staff  development  time.  (Dwyer,  1998). 

Rural  Needs 

Many  rural  educators  face  these  changes  with  the  added  challenge  of  geographic  isolation.  The  unique 
needs  of  rural  education  have  been  recognized  for  generations  (Leo-Nyquist  & Theobald,  1997).  When  the 
education  system  was  first  established  in  the  United  States,  technological  improvements  in  printing  and  distribution 
made  textbooks  and  educational  information  available  to  people  living  in  rural  areas.  Yankee  peddlers,  in  fact, 
influenced  the  content  of  textbooks  by  communicating  to  the  printers  the  specific  educational  needs  of  rural 
America,  (Smith,  1 993).  Distance  education  was  first  developed  for  use  by  farmers.  Courses  on  repairing  wagon 
wheels  were  offered  to  farmers  who  couldn’t  leave  their  fields.  (Thomas,  1999).  Beacham  & Kester,  (1994) 
identified  telecommunications  programs  as  useful  to  enhance  professional  growth  in  rural  areas  highlighting 
electronic  mentoring  as  a strategy  for  bridging  some  of  the  challenges  of  rural  isolation.  Today  advanced  technology 
is  able  to  provide  rural  communities  with  access  to  increased  educational  opportunities  and  information  vital  to 
quality  education  (Dennis,  1997). 

Ferre  and  Associates  (1988)  and  Kennedy  and  Barker  (1986)  identify  financial  and  funding  issues  as  the 
most  critical  issues  faced  by  small  rural  school  districts.  Since  nearly  a quarter  of  the  money  for  educational 
technology  comes  from  state  revenues,  funding  has  been  identified  as  a significant  problem  for  a large  segment  of 
the  country’s  educational  community.  Rural  districts  cannot  compete  with  urban  schools  in  the  area  of  funding 
simply  because  of  low  enrollments.  Corporations  seek  to  fund  programs  that  impact  the  largest  numbers  of  students. 
In  competing  for  grant  dollars,  rural  schools  are  at  a disadvantage  because  each  staff  member,  from  administration  to 
faculty,  is  having  to  wear  two  or  three  “hats”  (duty  assignments)  and  do  not  have  the  time  or  experience  to  write 
successful  grants  (Dennis,  1997).  So  funding  is  difficult  to  obtain,  technology  is  expensive  and  staff  development 
effectiveness  is  essential. 

Mentoring 

The  10-year  ACOT  project  (Apple  Classrooms  of  Tomorrow,  Saltpeter,  1998)  recommended  that  30 
percent  of  available  technology  resources  be  dedicated  to  providing  ongoing  staff  development.  Teachers  can 
improve  their  teaching  practices  by  engaging  in  frequent  and  planned  collaborative  activities  with  other  teachers. 
Such  activities  can  include  mentoring  (Becker,  1999).  Dwyer  (1998)  explores  the  theory  and  practice  of  mentoring. 
He  argues  that  on  -the-job  mentoring  has  the  potential  to  facilitate  critical  insight  into  the  changing  nature  of 
teachers’  work  and  to  transform  school  cultures.  Nellen  reported  that  teachers  participating  in  mentoring  activities 
became  confident  enough  about  using  technology  to  train  new  teachers,  with  a success  rate  of  100  percent  (Nellen, 
1999).  Mentoring  allows  the  learner’s  needs  to  define  the  experience  rather  than  following  a more  linear,  tutorial 
model  of  instruction  (Nellen  & Sweeny,  2000). 

Similar  research  indicates  that  teacher  mentoring  can  assist  faculty  members  and  new  teachers  with 
ongoing  support  and  provides  technology  integration  instruction  that  characterizes  effective  staff  development  (Mac 
Arthur  et.  al.,  1993:  Sprague  et  al.,  1998).  Anecdotal  evidence  from  a range  of  mentoring  projects  is  showing  that 
through  mentoring  more  significant  learning  occurs;  a safer  environment  for  risk-taking  is  developed  and  learning 
speeds  up. 

Mentors  benefit  from  an  increased  feeling  of  self-worth  and  mentees  from  increased  self-confidence.  Adler 
and  Harveil  (1996)  identified  several  benefits  of  mentoring  programs  that  included;  enhanced  recognition  of  the 
value  of  staff  development  and  teaming  to  make  effective  program  changes  and  improved  self-confidence  in  the 
receipt  of  support  and  encouragement.  Receiving  regular,  honest  and  constructive  feedback  and  being  part  of  a 
professional  network  is  mutually  beneficial.  Mentoring  is  seen  as  job-embedded,  ongoing  professional  development 
which  facilitates  long-term  change  and  transforms  workplaces  (Dwyer,  1998). 

Curriculum  Change 

There  is  great  deal  of  literature  in  the  field  on  curriculum.  Yet  there  is  little  empirical  evidence  that  focuses 
upon  attempts  at  curriculum  change,  where  school  districts,  schools,  or  particular  teachers  are  the  unit  of  analysis 
(Lewis,  1998).  Technology  implementation  requires  a well-designed  systemic  plan,  multiyear  funding,  and 
extensive  professional  development.  Teachers,  through  their  Internet  connections,  have  access  to  resources  that  only 
a few  years  ago  would  have  been  impossible  even  for  university  researchers.  Ninety  percent  of  all  teachers 
participating  in  Beckers  research  survey  (with  and  without  internet  connectivity)  ranked  internet  resources  as  either 
valuable  or  essential,  demonstrating  that  teachers  see  the  internet  as  a significant  resource  (Becker,  1999).  With  this 
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type  of  teacher  use,  sufficient  staff  development,  and  mentoring  support,  the  integration  of  technology  could  be  a 
supportive  tool  to  enhance  curriculum  and  the  learning  environment. 

In  the  Mills  study  findings  supported  the  position  that  teachers’  concerns  and  perceptions  of  technology 
influenced  the  way  in  which  they  implement  technology  (Mills,  1999).  The  Lecompte,  Millroy  and  Preissle  study 
(1992)  cited  three  common  themes  among  teachers  as  they  learned  technology  in  the  context  of  educational  practice; 
1)  changes  in  teaching  practice,  2)  changes  in  preparation  and  3)  increased  self-confidence  with  technology  use.  It  is 
crucial  therefore  to  integrate  technology,  pedagogy  and  application  into  competent  staff  development  programs  that 
demonstrate  effective  support  for  change  (Moore,  et.  al.,  1999). 

Collins  (1991)  describes  how  these  new  teaching/leaming  environments  differ  from  those  of  the  past  by 
citing  trends  identified  from  observations  of  schools  that  have  begun  using  technology.  The  shift  from  lecture 
based,  whole  group  instruction  to  coaching  student-directed  team-work  that  incorporates  technology  is  a major 
change  for  the  classroom  teacher  (Roblyer  & Edwards,  2000).  Well-developed  curriculum  can  be  the  directional 
force  that  organizes  and  promotes  technology  implementation  (Valdez,  et.al.1999).  Teachers  in  their  critical  roles  as 
“gatekeepers”  for  change  within  their  own  classrooms  and  schools  are  central  figures  in  curriculum  development 
and  change. (Leo-Nyqui st,  & Theobald,  1997). 

Methodology 

This  action  research  study  made  comparisons  between  a traditional  format  of  technology  staff  development 
to  an  onsite  team-mentoring  format.  The  traditional  format  of  staff  development  incorporated  two  2-  day  sessions  of 
hands-on  group  technology  instruction  at  the  service  center  facility.  The  mentoring  format  used  an  onsite  mentoring 
activity  that  included  development  of  integrated  classroom  projects  with  onsite  mentoring  for  collaborative  support. 
Participation  in  the  study  was  voluntary  and  school  sites  were  allowed  to  send  one  team.  Teams  were  self-selected 
and  four  schools  participated.  None  of  those  participating  in  the  mentoring  project  reported  previously  integrating 
technology  into  classroom  curriculum.  In  each  team,  the  member  who  felt  most  comfortable  with  technology  skills 
was  designated  as  mentor.  All  four  schools  were  located  within  a 100  mile  radius  of  the  service  center  facility, 
located  in  communities  with  a similar  agri-related  economy,  and  classified  1-A  (rural).  Grant  funding  was  budgeted 
to  include  upgrading  of  available  hardware  and  software  to  support  the  individual  projects  designed  by  the  teams. 
Funding  also  included  stipends  for  extra  planning  and  student  contact  hours. 

Data  collection  was  based  on  Profiler  scores,  observations,  interviews  and  anecdotal  information  collected 
by  the  technology  facilitator.  The  Profiler  is  a standardized  online  survey,  developed  by  South  Central  Regional 
Technology  in  Education  Consortium  (SCRTEC)  ( http:///profiler.scrtec.org/profiler/),  which  includes  30  questions 
that  assess  personal  confidence  in  technology  skill  and  application.  The  survey  was  taken  by  teachers  and  students 
participating  in  the  study  as  a pre  and  post  assessment  for  each  activity.  SCRTEC  is  one  of  six  Regional 
Technology  in  Education  Consortia  funded  by  the  U.S.  Dept,  of  Education,  whose  goal  is  to  help  teachers  and  other 
educators  create,  share,  or  find  solutions  to  problems  encountered  when  integrating  technology  into  education. 

Treatment 

Using  PowerPoint  or  HyperStudio,  students  created  presentations  based  on  personal  history.  After  initial 
instruction  concerning  the  content  of  the  social  studies  project  and  demonstrations  of  software  and  digital  camera 
application,  students  created  an  outline  of  personal  events  and  information  that  they  wished  to  include  in  their 
presentation.  Student  mentoring  teams  then  used  the  digital  camera  to  collect  pictures.  The  pictures  were  used  as 
information  focal  points  of  the  presentation.  Students  worked  on  formatting  skills  and  added  text  to  complete  the 
history.  Issues  of  vocabulary  and  editing,  as  well  as  the  creative  aspects  of  presentation  development,  were  all  part 
of  the  integrated  project.  Presentations  were  shared  in  a variety  of  ways  including  peer  review,  printed  hardcopy 
and  student  conferencing.  The  presentations  were  used  as  examples  of  authentic  assessment  of  student  technology 
skill  and  social  studies  goals  of  development  of  personal  identity,  culture  and  community  identity.  Three  school 
sites  participated  in  teacher  and  student  mentoring  activities.  Due  to  complications  with  teacher  re-assignment  one 
of  the  four  schools  did  not  participate  in  the  mentoring  activities. 

Results 

Concluding  data  indicated  that  all  teachers  and  students  completing  the  mentoring  activities  increased  on 
profiler  scores  indicating  that  self-reported  confidence  of  technology  skills  ability  increased.  In  comparing  teacher 
mentoring  team  scores  before  and  after  each  type  of  activity  (traditional  staff  development  and  mentoring  activity) 
data  show  that  teacher  scores  improved  21.33  percent  more  from  the  mentoring  activity  than  from  the  more 
traditional  staff  development  (Figure  1.1  & 1 .3).  This  is  three  times  the  gain  of  7.78  percent  made  after  traditional 
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staff  development.  While  traditional  staff  development  does  provide  positive  opportunity  for  teacher  improvement, 
the  addition  of  a mentoring  program  appears  to  greatly  enhance  the  effectiveness  of  technology  integration  training. 
When  comparing  groups  of  students,  those  participating  in  mentoring  activities  scored  over  1 1 percent  higher  than 
those  involved  with  technology  projects  without  mentoring  partners  (Figure  1 . 1 & 1 .4). 

1.1  Student  Perception  of  Comfort  With  Technology 


School  site 

1 

2 

3 

4 

Class  A Mentor 

5.93 

11.74 

8.19 

18.14 

(group  1) 

Class  B Mentored 

15.37 

6.89 

7.27 

18.14 

(group  2) 

Total  Avg.  Increase 

10.65 

9.32 

7.73 

18.14 

( School  sit  #3  completed  the  technology  project  without  mentoring ) 


1.2  Teacher  Perception  of  Comfort  with  Technology 


School  site 
Teacher 

1 

1 

2 

2 

1 

2 

3 

1 

2 

4 

1 

2 

l=mentee  2=mentor 
Pretest 

16.67 

X 

4.44 

22.22 

22.22 

51.11 

42.22 

24.44 

Staff  development 

20.00 

32.22 

57.77 

31.00 

25.56 

60  .00 

60.00 

30.00 

Posttest 

74.44 

48.89 

75.55 

55.56 

X 

X 

1.11 

57.78 

Total  Avg.  Increase 

57.77 

16.67 

21.11 

33.34 

3.34 

17.78 

38.89 

33.34 

(Note:  The  X indicates  that  there  was  no  profiler  score  and  in  both  cases  also  indicates 
that  the  teacher  was  not  involved  with  the  project  during  the  designated  time  of  evaluation.) 

School  site  #3  did  not  participate  in  mentoring  during  the  technology  project. 

1.3  Profiler  Average  Teacher  Change 

After  Staff  Development  8.46% 

After  Mentoring  Activity  29.2 1 % 

1 . 4 Profiler  A verage  Student  Change 

Without  mentoring  activity  7.73% 

Completing  mentoring  activity  1 8.75% 

Conclusion 

Final  evaluation  interviews  indicated  that  teachers  and  students  felt  the  project  mutually  beneficial. 
Having  a team  member  or  mentor  close  by  for  support  helped  to  promote  confidence  when  presenting  content  and 
having  students  work  with  technology.  Teachers  also  believed  that  having  a colleague/mentor  for  collaboration 
increased  their  ability  to  work  through  a variety  of  technical  problems.  Students  demonstrated  greater  interest  in 
content  and  were  more  involved  in  reflective  evaluation  to  improve  their  work.  Teachers  were  excited  by  the 
positive  responses  of  their  students  and  the  increased  motivation  they  observed.  The  key  goal  of  improving  student 
performance  was  realized  through  integration  of  a variety  of  technology  tools.  This  success  provided  strong 
encouragement  for  teachers  to  look  for  other  opportunities  to  integrate  technology. 

Research  project  results  mirrored  those  of  the  Nellen  study  (1999),  with  100  percent  of  teachers 
participating  in  the  mentoring  activity.  Teachers  indicated  that  they  felt  the  experience  so  valuable  they  intended  to 
continue  the  mentoring  format  during  the  next  year  by  instigating  new  technology  mentoring  projects.  Comments 
made  by  teachers  during  exit  interviews  provided  anecdotal  evidence  of  confidence.  One  teacher  said,  “I  never 
thought  that  I could  learn  to  use  technology  like  this.  Now  we  already  have  plans  for  another  mentoring  project  next 
year  and  my  students  are  asking  if  they  can  do  other  projects  with  the  computers.”  Another  teacher  enthusiastically 
remarked,  “I  saw  two  other  project  ideas  that  I’m  going  to  integrate  into  our  standard  curriculum  and  I’m  helping 
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another  teacher  make  plans  for  a project  for  next  year.”  Comments  such  as  these  document  the  increase  in  teacher 
confidence  integrating  technology  into  curriculum. 

The  use  of  mentoring  adds  a positive  support  for  technology  integration  in  the  classroom.  It’s  application 
is  inexpensive,  enhances  traditional  staff  development  and  promotes  positive  staff  team-building.  The  use  of 
mentoring  demonstrated  effectiveness  with  both  teacher  and  student  teams.  It  was  evident  from  observations  and 
interviews  that  that  mentoring  projects  between  classes  increased  student  interactivity  and  promoted  positive  school 
climate.  The  ability  of  mentoring  teams  to  discuss,  reflect  and  support  each  other  encouraged  risk-taking  and  self- 
improvement. 

Additional  study  in  the  area  of  mentoring  to  enhance  technology  integration  would  be  beneficial.  To  gain 
further  statistical  analysis  of  mentoring  application  it  would  be  valuable  to  design  research  that  would  control  for 
external  variables  such  as  grade  level  and  project  variation.  The  establishment  of  mentoring  teams  and  mentoring 
training  within  a rural  district  would  increase  the  effectiveness  of  teachers  as  resources  and  encourage  growth  and 
development.  Mentoring  encourages  a positive  teamwork  effort  to  promotes  educational  improvement. 
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Abstract 

One  useful  framework  for  understanding  and  articulating  instructional  design  to  change  biased  beliefs  is 
the  Martin  and  Reigeluth  model  for  affective  development,  appearing  in  Volume  II  of  Reigeluth ’s  Instructional- 
Design  Theories  and  Models  (1999).  Using  this  recently  developed  framework  to  examine  an  instructional  program 
aimed  at  affecting  biased  beliefs  suggested  that  the  addition  of  activity  components  and  the  language  of  activity 
theory  may  have  potential  to  extend  instructional  theory  on  affective  learning.  Consideration  of  action  as  both 
internal  and  external  enhances  the  model  in  a way  that  facilitates  design  and  evaluation  of  instruction  to  affect 
biased  beliefs  about  difference.  This  paper  examines  some  of  the  implications  of  including  the  concept  of  activity  in 
the  Martin  and  Reigeluth  model  for  affective  development. 


Introduction 

While  facilitating  affective  change  has  long  been  an  issue  in  efforts  to  impact  human  performance 
(Simonson,  1995),  it  has  most  often  played  a rather  cursory  role  in  the  development  of  instructional  systems  (e.g., 
Martin  & Reigeluth,  1999;  Kamradt  & Kamradt,  1999).  Moreover,  when  affect  is  addressed  in  the  design  of  learning 
and  performance  systems  it  is  often  done  so  as  an  aside  to  performance  outcomes  and  is  fragmented  as  a separate, 
less  significant  domain.  For  example,  the  Dick  & Carey  (1996)  model  of  systematic  design  addresses  learner’ 
attitudes  as  represented  by  behaviors  that  can  be  modeled  and  reinforced  relative  to  intended  performance  outcomes. 
This  model  relies  on  attention,  relevance,  confidence,  and  satisfaction  as  related  to  performance  goals.  Such  an 
approach  tends  to  fragment  internal  and  external  processes  and  may  serve  traditional  practice  while  actually 
constraining  design.  Until  recently  few  alternative  strategies  for  affecting  attitudes  existed  in  the  field  of 
instructional  systems  design  (ISD).  Whether  this  is  the  result  of  inadequate  paradigms,  a quest  for  certainty  (e.g., 
Dewey,  1929),  or,  as  Simonson  (1995)  suggests,  the  lack  of  a direct  causal  relationship  between  affect  and 
achievement,  is  of  less  concern  here  than  examining  the  potential  of  extending  a conceptual  model  to  guide  the 
design  of  instruction  aimed  at  affective  change. 

During  an  extensive  search  for  instructional  theory  to  guide  the  development  of  an  instructional  narrative 
simulation,  the  Martin  & Reigeluth  conceptual  model  emerged  in  ISD  literature,  offering  a helpful  tool  for  deeper 
thinking  about  affective  instruction.  An  analysis  of  the  instructional  simulation,  relative  to  this  model,  revealed,  as 
Martin  and  Reigeluth  indicate,  that  their  model  is  incomplete  (Reigeluth,  1999).  This  paper  presents  an  analysis  of 
an  instructional  multimedia  program  using  the  Martin  and  Reigeluth  conceptual  model.  Additionally,  it  attempts  to 
initiate  consideration  of  certain  aspects  of  activity  theory  as  a means  to  enhance  the  model  and  advance  development 
of  instructional  theory  on  affective  learning. 

Background  for  Analysis 

The  Martin  and  Reigeluth  conceptual  model  for  affective  development  is  used  here  to  examine  a web-based 
multimedia  instructional  program  aimed  at  changing  biased  beliefs  about  difference  in  sexual  orientation.  Jeffs 
Story  was  initially  designed  as  a paper  and  pencil  narrative  simulation  using  true  story  narrative  to  situate  the  learner 
as  a character  in  an  unfolding  story.  It  was  conceived  as  an  instructional  prototype  to  address  contemporary  social 
problems  of  bias  and  discrimination  based  on  a range  of  human  differences,  including  racism,  sexism,  cultural 
discrimination,  and  homophobia.  Bias  about  differences  in  sexual  orientation  became  an  initial  focus  because  it 
continues  to  be  acceptable  and  openly  expressed  in  contemporary  American  society  (e.g.,  Rhoades,  1994;  Tierney, 
1997).  Developing  a theoretical  framework  for  such  an  innovative  and  controversial  design  necessitated  reaching 
beyond  the  foundation  of  narrative  theory  (e.g.,  Britton  & Pellegrini,  1990;  Bruner,  1986,  1996;  Cole,  1997;  Fisher, 
1995;  Howard,  1991;  Polkinghorne,  1988;  Sarbin,  1986)  to  include  theory  on  situated  cognition,  anchored 
instruction,  and  case  study  (e.g.,  Anderson,  Reder,  & Simon,  1996;  Bliss  & Mazur,  1996;  Brown,  Collins,  & 
Duguid,  1989;  Cognition  and  Technology  Group;  1990;  Lave  & Wenger,  1991;  Mazur  & Bliss,  1995;  Young,  1993) 
culture  study  and  critical  social  theory  (e.g.,  Appiah  & Gutmann,  1996;  Ayers,  et  al.,  1998;  hooks,  1994;  Matsuda,  et 
al.,  1993;  Merry,  1990;  Peshkin,  1991;  Rhoades,  1994;  Tierney,  1997),  moral  education  (e.g.,  Kohlberg,  1971; 
Petrovic,  1999;  Vitz,  1990),  and  perception  and  aesthetics  (e.g.,  Dewey,  1934  ; Eisner,  1994  ; Greene,  1995).  These 


rich  theoretical  perspectives  were  combined  with  new  research  findings  on  issues  such  as  teaching  tolerance  with 
stories  and  arguments  (Colesante  & Biggs,  1999),  reactions  to  racist  hate  crimes  (Craig,  1999),  and  anti-gay 
behavior  (Franklin,  2000).  While  all  of  this  provided  important  foundation  for  continued  development  of  the 
instructional  program,  there  was  relatively  little  on  which  to  rely  in  ISD  until  Charles  Reigeluth  published  volume  II 
of  Instructional-Design  Theories  and  Models  in  1999.  The  conceptual  model  developed  by  Martin  and  Reigeluth  for 
this  volume,  along  with  subsequent  chapters  dealing  with  topics  ranging  from  attitude  development  to  character  and 
spiritual  development  (Kamradt  & Kamradt,  1999;  Lickona,  1999;  Moore,  1999)  offered  opportunity  to  examine 
Jeffs  Story  through  an  ISD  lens. 

Rationale  for  Consideration  of  Activity  Theory 

The  Martin  and  Reigeluth  model  includes  six  dimensions  of  affective  development  and  three  identified 
components  for  each  dimension.  The  dimensions  are  (1)  emotional,  (2)  moral,  (3)  social,  (4)  spiritual,  (5)  aesthetic, 
and  (6)  motivational.  The  components  of  each  dimension  include,  (a)  knowledge,  (b)  skills,  and  (c)  attitudes,  as  well 
as  a fourth  component  labeled  other  to  highlight  that  the  model  is  incomplete.  As  such,  it  invites  further 
development  and  encourages  research  and  theory  on  affective  learning  in  ISD.  For  example,  according  to  Martin  and 
Reigeluth,  emotional  development  can  be  viewed  as  ability  to  recognize,  express,  and,  as  appropriate,  control  one’s 
feelings,  emphasizing  understanding  and  managing  feelings  as  emotional  development.  Yet,  when  applied  to  the 
design  and  evaluation  of  one  instructional  narrative  simulation  to  change  biased  beliefs  this  aspect  of  the  model 
lacks  the  means  to  articulate  mediation  of  internal  and  external  emotional  activity.  Designing  instruction  aimed  at 
affective  learning  may  require  tools  that  ignite  internal  activity  in  ways  that  generate  reflective  responses  not  readily 
manifest  in  external  form.  Assessing  the  extent  of  actual  change  in  biased  attitudes  about  difference  highlights  the 
need  to  account  for  potentiality,  intent,  and  generative  change.  Activity  theory  makes  it  possible  to  explain  such 
affective  changes  without  reducing  complex  processes  to  measures  of  behavior  (Wertsch,  1998).  It  offers  concepts 
for  describing  human  activity  and  includes  the  central  notion  of  mediation  (e.g.,  Nardi,  1997;  Wertsch,  1998; 
Zinchenko,  1997)  that  can  be  extended  to  instructional  products.  Activity  theory  is  also  concerned  with  setting  or 
context  in  a manner  that  facilitates  consideration  of  social  construction  of  meaning,  which  is  so  crucial  to  design  for 
affective  learning.  In  these  ways  the  language  of  activity  theory  assists  in  broadening  the  scope  of  instructional 
design  aimed  at  affective  outcomes. 

Since  activity  theory  is  highly  complex  and  offers  no  concrete  methodology  or  procedural  guidelines,  it  is 
important  to  keep  in  mind  that  it  can  confound  design  efforts,  as  well  as  extend  design  considerations.  The  challenge 
here  is  to  incorporate  several  useful  concepts  of  activity  theory  without  losing  sight  of  the  focus  on  design  of 
instructional  products  for  affective  learning.  Therefore,  this  analysis  is  limited  to  the  use  of  a few  key  concepts  from 
activity  theory,  including  1)  activity  as  internal  and  external,  2)  tools  as  mediators  of  human  thought,  3)  thought  as 
potential  for  behavior,  and  4)  intent  as  central  to  affective  change. 

Application  Model 

Martin  and  Reigeluth  provide  an  application  model  based  on  the  conceptual  model  for  affective 
development.  Locating  Jeff’s  Story  in  this  application  model  provides  the  groundwork  for  further  analysis  (Figure 
1).  For  example,  Jeffs  Story  is  an  instructional  response  to  numerous  hate  crimes  in  the  United  States  and,  as  such, 
is  designed  to  be  both  preventive,  as  well  as  curative.  Perhaps  the  most  ambitious  goal  of  this  instructional  program 
is  the  opportunity  it  creates  for  users  to  begin  a process  of  embracing  diversity  in  a country  with  rapidly  growing 
ethnic  populations  and  major  shifts  in  majority  status.  Jeffs  Story  is  designed  as  an  isolated  instructional  event  for 
adults,  yet,  because  of  the  breadth  of  resources  that  users  may  access  throughout  the  exercise,  it  has  structural 
potential  to  be  interdisciplinary.  It  is  a “one-shot,,  experience  although  an  important  goal  of  the  program  is  to  ignite 
generative  reflection  and  continued  dialogue  and  therefore,  can  also  be  considered  a “spiraling”  learning  experience. 
For  instance,  if  users  take  away  from  the  experience  an  understanding  that  beliefs  are  not  always  based  on  factual 
information,  then  the  potential  exists  for  reconsideration  of  a range  of  beliefs  about  difference.  An  important 
component,  as  evidenced  in  user  testing  (McCrary  & Mazur,  1999),  is  a group  discussion  that  follows  the 
instructional  event,  offering  opportunity  for  learners  to  express  personal  understandings  in  public  conversation.  In 
this  way  the  program  is  both  internal  and  social  (intrapersonal  and  interpersonal)  to  the  extent  users  contribute  fully 
and  honestly  to  the  discussion.  The  instructional  methods  employed  in  the  narrative  simulation  are  direct  in  the 
sense  that  users  become  a character  in  an  unfolding  story  that  includes  dilemmas  to  which  they  are  asked  to  respond. 
The  methods  can  also  be  considered  indirect  in  efforts  to  engender  empathy  and  reflection. 
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Figure  1.  Adaptation  of  the  Martin  &Reigeluth  Application  Model  as  Applied  to  Jeff s Story,  positioned  in  the  Model 
with  a Triangle  on  each  Continuum. 
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The  Conceptual  Model 

The  design  of  Jeffs  Story,  as  it  incorporates  concepts  of  activity,  reintegrates  the  six  dimensions  of 
affective  development  while  primarily  focusing  on  social  and  emotional  change.  For  example,  it  includes  aesthetic 
mediators  in  the  form  of  original  music  and  images  in  an  effort  to  enhance  emotional  responses,  as  well  as  to  create 
interest  and  motivation.  Jeffs  Story  includes  a moral  dimension  by  presenting  possible  conflicts  in  an  effort  to 
create  cognitive  dissonance  (e.g.,  Piaget,  1970)  between  stereotypical  beliefs  and  ethical  standards.  Affective  design 
features  are  supported  by  didactic  information  in  the  form  of  historical  accounts,  news  articles,  and  hyperlinks  to 
websites  that  provide  additional  information. 

Analysis 

In  order  to  provide  a detailed  analysis  of  Jeffs  Story  as  related  to  the  Martin  and  Reigeluth  model  it  is  helpful  to 
examine  each  dimension  of  the  model  separately  as  it  is  represented  in  the  instructional  program.  Following  Table  1, 
which  shows  the  original  conceptual  model,  each  dimension  is  discussed  briefly  and  represented  in  Tables  2-7  as 


adapted  to  Jeffs  Story  in  the  language  of  activity  theory.  Each  dimension,  as  shown  in  Tables  2-7,  revolves  around  a 
major  instructional  goal  or  enhanced  ability  (e.g.,  Zinchenko,  1997),  including  empathy,  reflection,  imagination, 
aspiration,  prosocial  values,  and  democratic  principles.  It  is  important  to  keep  in  mind  that  Martin  and  Reigeluth 
focus  their  model  on  school-based  curricula,  while  Jeffs  Story  is  designed  for  adults  who  are  parents  or 
professionals  working  with  adolescents.  Tables  2-7  illustrate  an  examination  of  the  instructional  program  relative  to 
a)  design  components,  b)  mediating  elements,  and  c)  evaluation  data  for  each  affective  dimension.  While 
knowledge,  skills,  and  attitudes  are  represented  as  distinct  learning  activities  to  compliment  the  structure  of  the 
Martin  & Reigeluth  model,  it  is  important  to  remember  that,  from  an  activity  perspective,  these  domains  may  be 
inseparable. 

Table  1.  Martin  and  Reigeluth  Conceptual  Model  for  Affective  Development 


COMPONENTS  OF  INSTRUCTIONAL  VALUE 

DIMENSIONS 

Knowledge 

Skills 

Attitudes 

? 

Emotional 

Development 

Knowing  others  experience  the 
same  emotions  you  do,  such  as 
joy  and  anger. 

Recognizing  emotions 
and  controlling  one’s 
own  emotions. 

I want  to  be  happy.  I don’t 
like  to  be  angry. 

? 

Moral 

Development 

Understanding  moral  and  ethical 
rules  of  the  culture,  such  as 
caring,  justice,  and  equality. 

Moral  reasoning  and 
problem-solving  skills  in 
the  realm  of  morals. 

I want  to  be  honest  and  I 
am  in  favor  of  having 
ethical  standards. 

7 

Social 

Development 

Understanding  group  dynamics 
and  democratic  ideals. 

Social  skills,  including 
interpersonal 
communication  skills. 

I want  to  interact  positively 
with  others  and  am 
opposed  to  resolving 
disagreements  by  fighting. 

7 

Spiritual 

Development 

Knowledge  of  religious  precepts 
about  the  spiritual  world,  such  as 
the  nature  of  the  soul. 

Skills  for  getting  in  touch 
with  your  inner  self. 
Ability  to  love  others 
selflessly. 

I want  a spiritual  life  and 
am  in  favor  of  prayer  to 
build  a relationship  with 
God. 

? 

Aesthetic 

Development 

Understanding  the  subjective 
nature  of  aesthetics,  such  as  the 
relationship  between  one’s 
values  and  one’s  judgments. 

Skills  for  assessing 
aesthetic  qualities  and 
generating  aesthetic 
creations. 

I want  to  surround  myself 
with  things  of  beauty. 

? 

Motivational 

Development 

Understanding  internal  and 
external  rewards  for  sustained 
activity,  such  as  joy  and  sense  of 
accomplishment. 

Skills  for  developing 
one’s  interests,  both 
immediate  and  life-long. 

I want  a career  that  I enjoy. 

7 

Emotional  Development  as  a Dimension  of  Affective  Development 

Emotional  development  can  be  viewed  as  ability  to  recognize,  express,  and,  as  appropriate,  control  one’s 
feelings.  Individuals  can  move  through  feeling  states  without  associating  meaning  or  understanding  to  those 
emotions.  Martin  and  Reigeluth  emphasize  understanding  and  managing  feelings  as  emotional  development.  True 
story  narrative  simulation,  focused  on  emotionally  difficult  topics,  provides  experience  and  practice  in  identifying 
feelings  regarding  dilemmas  in  the  story. 
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Table  2.  Analysis  of  the  Instructional  Program  Relative  to  Emotional  Dimension. 


Empathy 

Affective  Learning  Activities  Relative  to  Emotional  Dimension 

Knowledge  Activity 

Skill  Activity 

Attitude  Activity 

Conceptual  Model  for 
Affective  Development 
(Martin  & Reigeluth) 

Knowing  others  experience  the 
same  emotions  you  do,  such  as 
joy  and  anger. 

Recognizing  emotions 
and  controlling  one’s 
own  emotions. 

I want  to  be  happy.  I don’t 
like  to  be  angry. 

Narrative  Simulation  to 
Affect  Biased  Beliefs 
about  Homosexuality 
(Jeffs  Story) 

Recognizing  inconsistencies 
between  long  held  beliefs  and 
new  information. 

Simulated  problem 
solving  as  a character 
in  an  unfolding  story. 

Desire  to  understand  why 
I feel  the  ways  I do  about 
homosexuality. 

Design  Components 

Back  Page  News. 

Historical  Accounts. 

News  Articles. 

Links  to  Advocacy  Groups. 

Unfolding  Narrative. 
Questions. 

Response  Choices. 
Group  Discussion. 

Perspectives. 

Concerns. 

Voices. 

Mediating  Elements 

User  controlled  navigation. 
Access  to  a variety  of 
information  presented  formally, 
informally,  and  as  personal 
statements. 

Story  Veracity. 
Fidelity  to  True  Story. 
Storied  Dilemmas. 

Images. 

Music. 

Authentic  Statements. 
Color. 

Evaluation  Data 

Computer  Tracking  Data. 
Program  Evaluation. 
Video  Data. 

Individual  Interview  Data. 

Computer  Tracking 
Data. 

Program  Evaluation. 
Observation  of  group 
discussion. 

Retrospective  Program 
Evaluation  Questions. 
Follow-up  Evaluation. 
Analysis  of  individual 
interview  data. 

Moral  Development  as  a Dimension  of  Affective  Development 

Although  moral  development  has  been  addressed  in  many  ways,  Martin  and  Reigeluth  describe  it  as 
building  codes  of  behavior  and  rationales  for  following  those  codes.  They  offer  as  an  example  having  positive 
attitudes  about  empathy  as  compared  to  understanding  or  experiencing  feelings  of  empathy,  which  are  components 
of  emotional  development.  This  area  of  affective  development  is  concerned  with  what  is  considered  right  and  wrong 
behavior  or  action  in  relation  to  issues  like  social  justice.  Although  Jeffs  Story  attempts  to  avoid  prescriptive  codes 
of  behavior,  it  offers  opportunity  for  moral  development  through  a process  of  decision  making  in  a simulated 
environment.  Jeffs  Story  also  provides  opportunities  to  identify  biases,  discuss  those  biases  with  others,  and  express 
rationales  for  personal  views  on  homosexuality.  The  sequencing  of  the  storied  narrative  follows  a pattern  of 
describing  particular  dilemmas  followed  by  questions  and  response  choices.  Each  response  choice  is  linked  to  a 
discussion  of  possible  consequences  as  a result  of  that  choice.  This  practice  of  problem  solving  is  designed  to 
stimulate  thinking  that  will  assist  users  in  identifying  their  own  beliefs  relative  to  each  situated  problem.  This 
unfolding  process  itself  has  the  potential  to  create  cognitive  dissonance,  which  in  turn  has  the  potential  to  stimulate 
reflection.  Such  reflection  may  be  a necessary  internal  activity  for  generating  serious  reconsideration  of  long  held 
beliefs.  In  terms  of  moral  development,  as  defined  by  Martin  and  Reigeluth,  Jeffs  Story  is  designed  to  stimulate 
examination  of  beliefs  about  homosexuality  by  juxtaposing  socially  constructed  beliefs,  one’s  own  situated  code  of 
ethics  or  morality,  and  disturbing  events  in  the  instructional  narrative. 
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Table  3.  Analysis  of  the  Instructional  Program  Relative  to  Moral  Dimension. 


Prosocial  Values 

Affective  Learning  Activities  Relative  to  Moral  Dimension 

Knowledge  Activity 

Skill  Activity 

Attitude  Activity 

Conceptual  Model  for 
Affective 
Development 
(Martin  & Reigeluth) 

Understanding  moral  and 
ethical  rules  of  the  culture, 
such  as  caring,  justice,  and 
equality. 

Moral  reasoning  and  problem- 
solving skills  in  the  realm  of 
morals. 

I want  to  be  honest 
and  I am  in  favor  of 
having  ethical 
standards. 

Narrative  Simulation 
to  Affect  Biased 
Beliefs  about 
Homosexuality 
(Jeffs  Story) 

Understanding  that  biased 
beliefs  are  based  (in  part) 
on  a social  norm  of 
heterosexuality. 

Using  independent  thinking  to 
resolve  conflict  between  moral 
stance  on  homosexuality  and  moral 
beliefs  about  respect  for  others. 

Desire  to  treat  all 
people  according  to 
ethical  standards. 

Design  Components 

Historical  Accounts 
News  Articles 
Statistical  Information 

Strategic  Questions 
Response  Choices 
Response  Choice  Discussions 

True  Story  Narrative 
Authentic  Personal 
Statements 

Mediating  Elements 

Juxtaposition  of  didactic 
information  on  particular 
viewpoints  of 
homosexuality  with  true 
story  narrative. 

Situating  the  user  as  a character  in 
an  unfolding  story. 

Disturbing  true  story 
problems  and 
outcomes. 

Evaluation  Data 

Computer  Tracking  Data 
Program  Evaluation 
Individual  Interview  Data 

Observation  of  group  discussion, 
comparison  of  tracking  data  with 
interview  transcripts  relative  to 
what  users  say  about  navigation 
through  resources  in  the  program. 

Observation  of  Group 
Discussion 
Retrospective 
Program  Evaluation 
Questions 

Individual  Interview 
Data 

Social  Development  as  a Dimension  of  Affective  Development 

Social  development  relates  to  having  and  maintaining  positive  relationships  with  others.  It  has  to  do  with 
valuing  relationships  and  distinguishing  positive  ways  to  promote  such  interactions.  Jeffs  Story  is  perhaps  focused 
more  on  social  development  than  any  of  the  other  dimensions.  It  offers  experience  and  information  designed  to 
promote  group  discussion  in  which  users  are  expected  to  engage  in  dialogue  with  others  to  share  perspectives  on 
homosexuality  and  discrimination. 


Table  4.  Analysis  of  the  Instructional  Program  Relative  to  Social  Dimension. 


Democratic  Principles 

Affective  Learning  Activities  Relative  to  Social  Dimension 

Knowledge  Activity 

Skill  Activity 

Attitude  Activity 

Conceptual  Model  for 
Affective  Development 
(Martin  & Reigeluth) 

Understanding  group 
dynamics  and  democratic 
ideals. 

Social  skills,  including 
interpersonal 
communication  skills. 

I want  to  interact  positively 
with  others  and  am  opposed 
to  resolving  disagreements  by 
fighting. 

Narrative  Simulation  to 
Affect  Biased  Beliefs 
about  Homosexuality 
(Jeffs  Story) 

Hearing  and  understanding 
the  views  of  others  on 
prevention  of  adolescent 
suicide  and  homosexuality. 

Identifying  and 
articulating  one’s  own 
perspectives  on 
homosexuality. 

Desire  to  maintain  positive 
interactions  with  all  others 
even  when  they  are  different 
or  disagree  with  my  views. 

Design  Components 

Historical  Accounts. 
Authentic  Statements. 
News  Articles. 

Back  Page  News. 
Links  to  Resources. 

Strategic  Questions. 
Response  Choice 
Discussions. 

Group  Discussion. 

Unfolding  Storied  Dilemmas. 
Authentic  Statements. 
Disturbing  Outcome. 

Mediating  Elements 

Juxtaposition  of  the  effects 
of  discrimination  with 
negative  beliefs  about 
homosexuality. 

Identifying  and 
articulating  one’s  own 
beliefs  regarding  same- 
sex  orientation. 

Storied  consequences  of 
discrimination  based  on 
sexual  orientation. 
Original  Music. 

Evaluation  Data 

Computer  Tracking  Data 
Program  Evaluation 
Group  Discussion  Data 
Individual  Interview  Data 

Observation  of  group 
discussion. 

Program  Evaluation 
Follow-up  Evaluation 
Individual  Interview  Data 

Spiritual  Development  as  a Dimension  of  Affective  Development 

Martin  and  Reigeluth  view  spiritual  development  as  distinct  from  religion.  It  is  concerned  with  awareness 
of  the  human  soul  or  spirit  and  the  interconnectedness  of  all  souls.  Specifically,  spiritual  development  relates  to  our 
ability  to  love  all  people  based  on  a concept  of  oneness,  according  to  these  authors.  JefPs  Story  is  directly  related  to 
this  dimension  in  the  sense  that  it  juxtaposes  universal  concerns  like  caring  for  an  adolescent  who  is  struggling  with 
a topic  that  many  feel  is  unrelated  to  their  own  circumstances.  It  situates  the  distant  other  in  the  typical  American 
home.  This  strategy  is  designed  to  encourage  users  to  imagine  the  homosexual  other  as  part  of  the  family  and  as 
spiritually  connected  as  anyone  else. 

Table  5.  Analysis  of  the  Instructional  Program  Relative  to  Spiritual  Dimension. 


Reflection 

Affective  Learning  Activities 

Relative  to  Spiritual  Dimension 

Knowledge  Activity 

Skill  Activity 

Attitude  Activity 

Conceptual  Model  for 
Affective 
Development 
(Martin  & Reigeluth) 

Knowledge  of  religious 
precepts  about  the  spiritual 
world,  such  as  the  nature  of 
the  soul. 

Skills  for  getting  in 
touch  with  your  inner 
self.  Ability  to  love 
others  selflessly. 

I want  a spiritual  life  and  am  in 
favor  of  prayer  to  build  a 
relationship  with  God. 

Narrative  Simulation 
to  Affect  Biased 
Beliefs  about 
Homosexuality 
(Jeffs  Story) 

Understanding  that 
religious/spiritual  beliefs  are 
constructed  through  past 
experiences  and  cultural 
influences. 

Distinguishing  truth 
from  stories  held  as 
truth  or  religious 
doctrine. 

Desire  to  resolve  conflicts 
between  religious  beliefs, 
personal  spirituality,  and  new 
feelings  arising  from  simulated 
experience  as  the  parent  of  a gay 
adolescent. 

Design  Components 

True  Story  Narrative. 
Historical  Accounts. 
News  Articles. 
Authentic  Personal 
Statements. 

Storied  Dilemmas. 
Strategic  Questions. 
Response  Choices. 

Veracity  of  the  narrative  that 
assists  users  in  imagining  the 
adolescent  in  the  story  as  their 
own  son. 

Mediating  Elements 

Storied  juxtaposition  of  a 
typical  adolescent  who  also 
happens  to  be  gay. 

Making  difficult 
decisions  as  a parent 
of  the  adolescent  in  the 
story. 

Confronting  attitudes  of  others 
and  recognizing  one’s  own 
through  group  interaction. 

Evaluation  Data 

Computer  Tracking  Data. 
Program  Evaluation. 
Observation. 

Individual  Interview  Data. 

Computer  Tracking 
Data. 

Program  Evaluation. 
Group  Observation. 

Retrospective  Program. 
Evaluation  Questions. 
Follow-up  Evaluation. 

Aesthetic  Development  as  a Dimension  of  Affective  Development 

Aesthetic  development  is  concerned  with  appreciation  of  beauty,  which  includes  recognition,  creation,  and 
valuing  aesthetic  qualities.  It  is  a development  of  sensitivity  to  one’s  internal  and  external  environments.  Aesthetics 
include  organization  of  space,  time,  and  thought  in  ways  that  are  pleasing,  ways  that  are  congruent,  unified, 
balanced,  and  interesting.  It  relates  to  cognitive  dissonance  theory  in  the  sense  that  human  beings  may  naturally  seek 
balance  and  consonance  in  their  minds,  as  well  as  in  their  surroundings.  Beauty  is  in  the  mind  of  the  beholder  by 
virtue  of  the  level  of  aesthetic  development  achieved.  The  design  of  Jeffs  Story  employs  narrative  and  other 
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aesthetic  mediators  as  central  to  affective  instruction.  These  mediators  include  original  art,  chosen  for  qualities  that 
have  potential  to  enhance  and  extend  the  storied  topics,  original  flute  music,  composed  specifically  for  each  story 
theme,  and  a variety  of  compositional  elements  to  engender  interest  and  facilitate  the  process  of  making  meaning  of 
the  events  in  the  program.  Since  this  instructional  design  and  development  project  focuses  on  facilitating  enhanced 
perception  and  stimulating  reflection,  concern  for  aesthetics  is  viewed  as  essential  in  mediating  internal  activity  that 
may  lead  to  new  perspectives. 

Table  6.  Analysis  of  the  Instructional  Program  Relative  to  Aesthetic  Dimension . 


Imagination 

Affective  Learning  Activities  Relative  to  Aesthetic  Dimension 

Knowledge  Activity 

Skill  Activity 

Attitude  Activity 

Conceptual  Model 
for  Affective 
Development 
(Martin  & 
Reigeluth) 

Understanding  the 
subjective  nature  of 
aesthetics,  such  as  the 
relationship  between  one’s 
values  and  one’s 
judgments. 

Skills  for  assessing  aesthetic 
qualities  and  generating 
aesthetic  creations. 

I want  to  surround  myself 
with  things  of  beauty. 

Narrative 
Simulation  to 
Affect  Biased 
Beliefs  about 
Homosexuality 
(Jeffs  Story) 

Understanding  the 
subjective  nature  of  beliefs 
about  homosexuality  and 
the  opportunity  afforded 
by  subjectivity  to  imagine 
new  ways  of  viewing 
same-sex  orientation. 

New  appreciation  of  the  beauty 
and  function  of  human 
diversity,  imagining  positive 
ways  to  interact  with 
homosexuals  or  others  who  are 
different  in  some  way  from 
oneself,  and  recreating  one’s 
own  stories  about  same-sex 
orientation. 

Desire  to  enrich  and  enhance 
one’s  life  through 
interactions  with  those  who 
are  different  from  one’s  self 
and  appreciate  the  subjective 
nature  of  diverse 
perspectives,  as  well  as 
one’s  own  subjectivity. 

Design 

Components 

Story  Themes. 
Response  Choices. 
Response  Choice 
Discussions. 
Historical  Accounts. 
Voices. 

Perspectives. 

Back  Page  News. 
Concerns. 

Consonant  and  dissonant 
patterns  in  the  narrative, 
imagery,  and  music. 

True  Story  Narrative. 
Perspectives. 

Back  Page  News. 
Response  Choice 
Discussions. 

Mediating 

Elements 

Images. 

Original  music,  composed 
to  reinforce  story  themes. 
Simulated  role  as  an  actor 
in  the  in  the  story. 

Solving  problems,  selecting 
possible  responses,  and  reading 
contrasting  excerpts  of  other’s 
stories  that  inspire  thinking 
about  the  subjective  nature  of 
that  based  on  stories  heard  and 
remembered  in  ways  that 
reinforce  one’s  own  beliefs  and 
comfort. 

Veracity. 

Fidelity. 

Imagery. 

Original  music,  composed  to 
reinforce  story  themes. 
Group  Interaction. 

Group  Discussion. 

Evaluation  Data 

Program  Evaluation. 
Response  Choices. 
Individual  Interview  Data. 
Group  Discussion  Data. 

Computer  Tracking  Data. 
Program  Evaluation. 
Individual  Interview  Data. 
Group  Discussion  Data. 

Retrospective  Program 
Evaluation  Questions. 
Individual  Interview  Data. 
Group  Discussion  Data. 

Motivational  Development  as  a Dimension  of  Affective  Development 

Developing  one’s  own  interests  is  the  major  concern  in  motivational  development.  Preference,  choice, 
focus,  and  navigation  are  all  elements  of  Jeffs  Story  that  acknowledge  personal  style  and  facilitate  individual 
interests.  While  this  instructional  program  is  not  specifically  designed  to  promote  motivational  development  it  offers 
opportunity  to  identify  preferences  and  navigate  according  to  one’s  particular  curiosity.  Computer  generated 
tracking  data  is  used  partially  to  provide  insight  into  the  motivations  and  interests  of  learners. 
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Table  7 . Analysis  of  the  Instructional  Program  Relative  to  Motivational  Dimension . 


Aspiration 

Affective  Learning  Activities  Relative  to  Motivational  Dimension 

Knowledge  Activity 

Skill  Activity 

Attitude  Activity 

Conceptual  Model 
for  Affective 
Development 
(Martin  & Reigeluth) 

Understanding  internal  and 
external  rewards  for 
sustained  activity,  such  as 
joy  and  sense  of 
accomplishment. 

Skills  for  developing  one’s 
interests,  both  immediate  and 
life-long. 

I want  a career  that  I enjoy. 

Narrative  Simulation 
to  Affect  Biased 
Beliefs  about 
Homosexuality 
(Jeffs  Story) 

Understanding  personal 
and  social  benefits  of 
diversity. 

Deconstructing  one’s  own 
stories  about  people  who  are 

gay. 

Desire  to  interact  in  positive 
ways  with  homosexuals  or 
others  who  are  different  from 
one’s  self  and  feeling  that 
such  interaction  will  enrich 
one’s  life. 

Design  Components 

Storied  Dilemmas. 
Response  Choices. 
Response  Choice 
Discussions. 

Simulated  exposure  to  a 
variety  of  attitudes,  beliefs, 
and  information  about 
homosexuality. 

User  Controlled  Navigation. 
Historical  Accounts. 

News  Articles. 

Links  to  and  Descriptions  of 
Advocacy  Groups. 

Concerns. 

Mediating  Elements 

Reflection  due  to  simulated 
experience  with  the 
perspectives  of  others. 

Immediate  feedback  on 
response  choices  in  the  form 
of  discussions  of  possible 
consequences  associated  with 
particular  choices  of  action. 

Stimulated  experience  in  the 
role  of  a parent  of  the 
adolescent  in  the  story. 

Evaluation  Data 

Program  Evaluation. 
Individual  Interview  Data. 

Selected  Response  Choice 
Data. 

Retrospective  Program 
Evaluation  Questions. 
Individual  Interview  Data. 
Group  Discussion  Data. 

Computer  Tracking  Data. 
Retrospective  Program 
Evaluation. 

Observation  of  Group 
Interaction. 

Video  Data  of  Group 
Discussion. 

Research  Implications 

The  integration  of  several  key  concepts  from  activity  theory  with  the  Martin  & Reigeluth  model  serves 
research,  as  well  as  design,  in  developing  instruction  aimed  at  bias.  For  example,  the  conceptual  model  describes 
what  learners  will  be  able  to  do  as  a result  of  instructional  intervention,  while  activity  theory  turns  attention  to  the 
mediation  of  internal  and  external  to  create  opportunity  for  new  activity.  Consideration  of  mediating  internal  and 
external  activity  provides  opportunity  for  asking  qualitatively  different  research  questions  that  have  more  to  do  with 
how  the  process  of  affective  change  evolves.  Relative  to  Jeffs  Story,  it  is  helpful  to  examine  each  dimension  of 
affect  (Martin  & Reigeluth,  1999)  through  mediating  processes.  Aesthetic  change,  for  example,  includes  1) 
perception  of  the  storied  nature  of  beliefs,  2)  imagination  of  what  can  be,  3)  appreciation  for  the  beauty  of  and 
necessity  for  human  diversity.  The  questions  evolve  from  how  to  promote  appreciation  of  beauty  and  style  (Martin 
& Reigeluth,  p.  94)”  to  more  specific  inquiry  into  perception  and  imagination.  Table  8 frames  Jeffs  Story  relative  to 
mediating  activities  of  instructional  goals. 
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Table  8.  Mediating  Activities  for  Desired  Instructional  Outcomes  for  Jeffs  Story . 


Dimension 

Mediating  Activity 

Desired  Instructional  Outcomes 

Perception 

of  the  storied  nature  of  beliefs. 

Aesthetic 

Imagination 

of  what  can  be. 

Appreciation 

for  the  beauty  and  necessity  of  diversity. 

Social 

Articulation 

of  biased  beliefs  in  a group  setting. 

Emotional 

Empathy 

for  the  feelings  and  experiences  of  others. 

Tolerance 

for  the  differences  of  others. 

Spiritual 

Self  reflection 

on  the  nature  and  origins  of  one’s  beliefs  about  others. 

Connectedness 

to  all  human  beings. 

Motivational 

Value 

for  diversity. 

Desire 

to  be  tolerant. 

Moral 

Principles 

guiding  behavior  towards  others. 

Fairness 

in  relation  to  issues  of  social  justice. 

Summary 

As  apparent  in  both  the  Martin  and  Reigeluth  model  and  the  analysis  of  Jeffs  Story,  instructional  priorities 
for  affective  outcomes,  as  opposed  to  performance  or  behavioral  goals,  may  require  new  ways  of  approaching 
design  and  research.  Including  certain  concepts  of  activity  theory  extends  consideration  of  learning  as  integrated 
internal  and  external  processes  that  are  facilitated  by  mediating  processes  like  perception,  imagination,  and 
reflection.  This  hybrid  approach  prioritizes  mediation  over  information,  feelings  over  facts,  and  meaning  over 
certainty.  Traditional  concern  for  structure  and  sequence  of  instruction  is  transformed  to  artful  composition  of 
learning  environments  that  provide  opportunity  and  potential  for  new  representations.  Such  an  approach  to  affective 
learning  may  actually  promote  a broader  understanding  of  human  development,  integration  of  interdisciplinary 
perspectives,  and  understanding  design  as  the  composition  of  elements  in  space  and  time  to  promote  learning. 
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Abstract 

A person-centered  model  of  instruction  has  been  developed  for  use  in  designing  instruction  in  virtual,  web- 
based  environments.  This  model,  based  on  the  work  of  Carl  Rogers  attempts  to  address  several  issues  raised  in  the 
literature  regarding  a)  the  changing  role  of  instructors  and  students  b)  the  broadening  of  the  notion  of  learning 
outcomes,  c)  the  isolation  and  dissatisfaction  of  students  in  dispersed  locations  and  d)  problems  with  authenticity 
and  individualization  of  experience.  A person-centered  instructional  model  is  described  and  contrasted  with 
instructionist  (Dick  and  Carey)  and  constructivist  (Jonassen)  approaches. 

Introduction 

Virtual,  web-based  environments,  while  attractive  in  their  potential  to  widen  the  scope  of  users’ 
experiences  for  communication,  collaboration,  and  access  to  resources  can  also  create  artificial  and  possibly 
depersonalizing  social  circumstances.  A person-centered  model  of  instruction  has  been  developed  for  use  in 
designing  instruction  in  virtual,  web-based  environments.  This  model,  based  on  the  person-centered  theory  of  Carl 
Rogers  attempts  to  address  several  issues  raised  in  the  literature  regarding  a)  the  changing  role  of  instructors  and 
students  b)  the  broadening  of  the  notion  of  learning  outcomes,  c)  the  isolation  and  dissatisfaction  of  students  in 
dispersed  locations  and  d)  problems  with  authenticity  and  individualization  of  experience.  How  should  instructors 
confront  these  types  of  issues  which  arise  in  a distributed  instructional  network? 

In  this  paper  we  posit  that  Rogers’  work  can  be  used  to  design  instruction  for  virtual  web-based 
environments  and  we  offer  a conceptual  analysis  upon  which  to  base  this  claim.  A person-centered  instructional 
model  is  described  and  contrasted  with  instructionist  (Dick  and  Carey)  and  constructivist  (Duffy  and  Jonassen) 
approaches.  A discussion  of  problems  and  potentials  concludes  the  article. 

Defining  Virtual  Environments 

As  with  any  new  technology,  definitions  that  are  complementary,  mutually  exclusive,  subsuming,  and 
directly  contradictory  seem  to  proliferate  as  fast  as  the  innovation  itself.  Virtual  reality  and  its  many  applications  are 
no  exception.  In  this  section  we  give  an  overview  of  prominent  definitions  and  select  one,  to  be  used  for  purposes  of 
analysis  in  this  article.  Virtual  reality  (VR)  is,  quite  literally,  an  analogous  reality  to  our  own  but  with  one  significant 
difference— it  occurs  in  computerized  and/or  networked  electronic  environment.  Definitions  of  VR  range  from  " one 
part  computer  simulation  and  one  part  consensual  hallucination"  (Biocca  & Levy,  1995;  Gibson,  1984  p.54)  to  a 
computer-created  sensory  experience  completely  immersing  a participant  so  they  believe  and  barely  distinguish  a 
“virtual”  experience  from  a real  one  (Franchi,  1994). 

Types  of  VR  applications  tend  to  fall  into  two  general  categories:  immersion  and  simulated  environments. 
VR  immersion  environments  using  specially  designed  hardware  worn  by  the  user  can  literally  make  the  individual 
feel  like  they  are  in  another  environment  in  a cyberspace.  In  contrast,  in  VR  simulated  environments,  the  user 
experiences  a particular  context  or  situation  in  a much  less  sensory  and  more  cognitive  way,  by  accessing  software 
applications  on  a networked  hyperspace  — such  as  the  World  Wide  Web.  These  types  of  WWW  virtual 
environments  have  several  common  applications.  Users  can  take  virtual  tours  of  on-line  museums  or  other  remote 
locations  such  as  strolling  around  virtual  parks  or  navigating  to  selected  locations  via  a virtual  map  of  particular 
areas.  Another  common  VR  simulation  environment-  a chat  room--  provides  tools  for  participants  to  talk  with 
people  at  geographically  dispersed  locations  in  real  time  conversations.  Since  simulated  VR  environments  require 
less  hardware  and  technical  commitment,  they  have  tended  to  flourish  in  educational  settings. 

In  fact,  according  to  Mason  (1996),  the  many  virtual  classrooms  and  universities  can  be  characterized  by 
three  broad  categories: 

1.  Text  based  systems,  including  electronic  mail,  computer  conferencing,  real  time  chat  systems, 

MUDs/MOOs,  and  other  WWW  applications; 

2.  Audio  conferencing  such  as  audiographics,  and  real  time  audio  over  the  Internet;  and 
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3.  Videoconferencing,  one-way  and  two-way,  software  driven  videoconferencing  and  other  web-based 

visual  media. 

Virtual  education  purportedly  differs  from  traditional  education  in  less  obvious  ways  than  the  presentation 
mode.  Chalmers  (1997)  asserts  that  a virtual  educational  space  can  offer  increased  levels  of  interactivity  and  the 
development  of  learning  communities  through  the  use  of  the  communication  tools  described  by  Mason.  One 
example  of  this  increased  activity  level  is  apparent  in  a text-based  interactive  learning  environment,  PuebloMOO. 
PuebloMOO  (http://www.pc.maricopa.edu/community/pueblo/)  is  a complex  environment  where  the  students  are 
free  to  explore  a world  created  completely  in  the  computer,  interact  with  other  people,  and  make  choices  regarding 
the  character  they  use  (called  an  avatar)  in  the  virtual  environment.  The  opportunity  to  personalize  one’s  role  in  a 
virtual  environment  could  be  beneficial  because  it  allows  learners  to  meld  learning  with  recreation  and  socialization 
developing  activities  all  at  one  time. 

Virtual  environments,  while  attractive  in  their  potential  to  support  a variety  of  users’  interactions,  also  have 
the  potential  to  create  artificial  and  possibly  depersonalizing  social  circumstances. 

Dehumanizing  Effects  of  Virtual  Environments 

Concerns  associated  with  the  dehumanizing  effects  of  mass-produced,  one-size-fits-all  instruction  is  framed 
by  the  largely  European  debate  of  Fordist,  Neo-Fordist  and  Post-Fordist  approaches  (Campion,  1995).  On  a 
continuum  from  maximum  central  control,  low  skill,  and  little  learner  responsibility  (Fordism)  to  less  managerial 
control,  high  skill  and  responsibility  for  learning  (Post-Fordism),  each  position  implies  that  control  emanates  from 
the  point  of  instruction  (or  the  instructor)  and  is  not  shaped  through  negotiations  between  the  instructor  and  learner. 
In  other  words,  though  the  debate  addresses  issues,  which  arise  from  the  social,  the  underlying  epistemological 
assumptions  of  the  positions  specifically  exclude  the  social  negotiation  of  learning.  While  this  theoretical  debate  is 
less  publicly  articulated  in  the  United  States,  it  is  important  to  note  how  often  disgruntled  distance  education 
students  report  they  feel  alienated  and  dissatisfied  with  on-line  learning  (Biner,  Dean,  & Mellinger,  1994).  Students 
at  the  so-called  "remote  sites"  complain  of  a lack  of  co-presence  with  the  instructors  and  other  students.  In  fact,  the 
only  consistently  reported  benefit  is  "convenience."  While  American  educators  claim  interaction  and  personalized 
instruction  are  valued  (Simonson,  Smaldino,  Albright,  & Zvacek,  2000  p.  41),  reports  of  distance  learning 
experiences  show  these  goals  less  often  achieved  in  actual  practice.  How  should  instructional  designers  use  and 
possibly  modify  available  theory  and  concepts  to  maximize  the  potential  of  virtual  environments  while  confronting 
the  personalization  issues  that  seem  to  develop  in  a distributed  network  of  instruction? 

In  search  of  theory-based  instruction  for  Virtual,  Web-based  Environments 

The  introduction  of  new  interactive  technologies  inevitably  effects  the  learning  context.  In  virtual 
environments,  both  the  role  of  the  instructor  and  the  form  of  instruction  will  change  dramatically.  A central  change 
will  be  less  control  for  the  instructor  coupled  with  more  opportunities  for  learner-selected  and  -controlled 
exploration  and  interaction.  Indeed,  unbeknownst  to  the  instructor  at  any  given  point  in  time,  students  may  be 
logging  in  and  participating  actively.  Directive,  didactic  forms  of  instruction  will  need  modification.  Instruction  that 
is  shaped  and  enhanced  by  facilitation  may  be  a key  to  accommodating  increased  learner  control.  Students  will 
clearly  need  specialized  guidance  exploring  their  on-line  learning  opportunities  and  the  design  of  instruction  will 
need  to  take  into  account  the  special  nature  asynchronous  interaction  supported  by  web-based  virtual  environments. 
We  hypothesize  the  notion  of  instructional  design  as  primarily  facilitation  can  be  informed  by  the  work  of  Carl 
Rogers. 

When  Rogers  wrote  Freedom  to  Learn  (1994),  he  was  focused  on  traditional  schools  but  saw  the  person- 
centered  educational  approach  developing  its  strongest  roots  in  alternative  schools  and  what  he  presciently  called 
"universities  without  walls."  While  many  instructional  theories  focus  on  the  learner's  achieving  specific  learning 
objectives,  Rogers'  instructional  theory  focuses  on  a goal  of  teaching  the  learner  how  to  learn.  It  is  because  of  this 
focus  that  Rogers  felt  the  learner  would  become  a freely  functioning,  self-enhancing,  self-actualizing,  creative,  and 
dependable  person. 

Carl  Rogers  revealed  that  he  developed  his  person-centered  theory  because  we  live  in  a constantly  changing 
world  and  that  people  in  such  continually  evolving  contexts  needed  to  be  flexible  thinkers  adapting  easily  to  change. 
More  importantly,  he  claimed,  the  constant  of  change  required  students  to  learn  how  to  learn  to  adapt  to  different 
types  of  learning  required  in  a variety  of  settings  for  myriad  purposes.  Rogers  boldly  suggested  the  facilitator  should 
encourage  the  learners  to  charge  off  in  new  directions  dictated  by  their  own  interests  and  to  unleash  their  sense  of 
inquiry  and  exploration  (Rogers  & Freiberg,  1994).  How  can  the  design  of  instruction  support  such  learner  directed 
activity?  What,  if  any,  will  be  the  outcomes  of  such  instruction  and  how  can  evaluation  take  place?  Will  assessment 
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be  mutual  or  individual?  What  kinds  of  performance  are  desirable?  Will  learner  development  and  satisfaction  lead  to 
outcomes  that  will  be  valued  personally  and  publicly? 

The  purpose  of  this  paper  is  to  discuss  the  utility  of  the  person-centered  instructional  theory  of  Carl  Rogers 
for  designing  instruction  in  virtual  environments  based  on  both  a conceptual  and  a comparative  analysis.  The 
conceptual  analysis  focuses  on  outcomes  ascribed  to  web-based  learning  from  the  literature  as  they  can  be  framed  by 
specific  aspects  Rogers’  theory  such  as  personal  development  and  learners’  self  actualization.  Next,  we  present  a 
model  of  person-centered  instruction  and  compare  it  to  two  prominent  instructional  approaches.  We  follow  this 
discussion  with  a case  example  of  a course,  which  applies  aspects  of  the  model.  Finally,  a discussion  of  problems 
and  potentials  with  the  model  concludes  the  article. 

A conceptual  Analysis  of  Rogers’  Person-Centered  Instructional  Theory 

Using  an  Epistemological  Heuristic 

We  posit  that  the  foundational  conceptual  analysis  for  our  investigation  of  the  utility  and  robustness  of 
Rogers’  work  as  it  may  be  applied  to  on-line,  virtual  learning  environments  must  begin  with  a sound  epistemological 
analysis  of  his  theoretical  perspective.  In  order  to  accomplish  this  goal,  we  employed  the  “structure  of  knowledge” 
approach  developed  by  Gowin  (1981).  This  approach  is  particularly  appropriate  because  it  is  inquiry-based  — the 
analysis  proceeds  from  central  questions  emanating  from  a learning  event  (in  our  case  the  event  is  “instruction  in  a 
web-based  environment).  Our  central  “focus  questions”  are  “what  are  the  epistemological  elements  (world  view, 
principles,  concepts)  of  Rogers’  theory?”  and  “how  can  these  be  applied  to  instructional  design  in  virtual 
environments?”  Gowin ’s  heuristic  details  several  components  of  the  underlying  knowledge  structure  of  a given 
theoretical  approach  such  that  world  view,  core  principles  and  concepts  are  related  to  directly  to  a specific  learner 
event  and  the  various  knowledge  claims  and  value  to  the  learner  can  be  described.  The  results  of  this  analysis 
follow. 

Carl  Rogers  and  Person-Centered  Learning 

Carl  Rogers  developed  a system  of  non-directive  psychology  called  client-centered  therapy  that  allows  the 
client,  who  knows  what  hurts,  to  marshal  the  resources  of  personal  experiences  and  discover  their  own  meanings. 
The  client  learns  through  such  reflective  experience  and  uses  it  to  grow  as  a person.  Rogers  hypothesized  that  core 
concepts  of  client-centered  psychology  could  be  applied  analogously  towards  a person-centered  education. 

Rogers  theorized  that  a person  emerging  from  therapy  or  from  the  best  of  education  has  experienced 
optimal  psychological  growth  (Rogers  & Freiberg,  1994).  Specifically,  the  person  is  able  to  function  freely  — 
realizing  his  or  her  potentials,  striving  to  be  self-enhancing,  continuing  to  develop,  and  always  seeking  newness  in 
each  moment--  resulting  in  a self-actualized  person.  Maslow  (1970),  describes  this  self-actualized  person  as 
someone  who  has  developed  or  is  developing  into  the  full  stature  of  personal  capability.  Of  importance  is  that 
learners  continue  to  learn  creatively  through  life  rather  than  becoming  automatons  reciting  the  information  provided 
to  them  (Patterson,  1973).  Rogers  himself  tied  self-actualization  to  creativity  with  these  words:  "The  mainspring  of 
creativity  appears  to  be  the  same  tendency  which  we  discover  so  deeply  as  the  curative  force  in  psychotherapy  - 
one's  tendency  to  actualize  oneself,  to  become  one’s  potentialities  ...  the  urge  to  expand,  extend,  develop,  mature  - 
the  tendency  to  express  and  activate  all  the  same  capabilities  of  the  organism  ..."  (Davis,  1992  pp.  3-4).  Building 
on  Rogers,  Davis  claims  creativity  involves  developing  your  talents;  learning  to  use  your  abilities;  exploring  new 
ideas,  places,  activities;  and  developing  a sensitivity  to  problems  of  others  and  humankind  (Davis,  1992  p.  7).  In  a 
person-centered  approach  the  linking  of  self-actualization,  freedom,  and  creativity  will  be  required  in  order  to  design 
instruction,  which  accommodates  the  largely  unfettered,  learner-controlled  choices  available  to  a user. 

Davis'  (1992)  elaborated  the  notion  of  creativity  in  terms  of  what  he  called  the  "4-P's":  The  creative  person, 
process,  product  (Barron,  1988),  and  press  (Isaksen,  1987;  Mooney,  1963;  Taylor,  1988).  The  creative  person  is  the 
individual  in  the  creative  environment,  moving  through  the  process  of  creativity,  or  having  created  the  creative 
product  (Davis,  1992).  The  creative  process  is  the  steps  taken  to  creatively  solving  real  problems  (Davis,  1992). 
The  creative  product  is  the  outcome  of  the  creative  process.  It  can  emphasize  originality,  and  a sense  of  value 
(Davis,  1992).  Creative  press  (as  in  pressure)  is  the  social  and  psychological  environment  affecting  any  other  aspect 
of  the  creative  person,  process,  or  product  or  all  three. 

While  the  development  of  self-actualized  creative  people,  who  are  life-long  learners,  is  clearly  a 
commendable  goal,  how  can  this  goal  be  actually  achieved?  Firstly,  an  instructor  should  realize  self-actualization 
cannot  be  taught  and  the  student  reaches  the  goal  of  becoming  a self-actualized  person  through  his  or  her  own 
individualized  learning  experience.  Unlike  didactic  teaching  methods,  which  provide  knowledge  to  the  learners,  the 
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teacher  in  a person-centered  environment  becomes  a facilitator  of  the  learning  the  students  conduct.  Such  a 
facilitated  experience  is  termed  "significant  learning"  because  the  individual  initiates  it,  allowing  the  individual  to 
provide  personal  control  with  the  element  of  learning  built  into  the  whole  experience  (Sahakain,  1970).  In  this 
experience  the  learner  becomes  "the  creative  person"  engaged  in  a creative  process.  There  are  several  tasks  for  an 
instructor  wishing  to  move  into  a facilitator  role.  The  teacher  should  first  set  the  mood  for  the  environment  or  the 
creative  press.  There  should  be  a sense  of  cooperation  and  trust  within  the  group  enhancing  the  creative  experience 
rather  than  competitive  attitudes  which  will  disrupt  the  sense  of  trust  and  cooperation  thus  creating  a negative 
creative  press  on  the  experience.  Next,  the  teacher  as  facilitator  becomes  one  of  many  resources  of  information 
rather  than  as  the  main  source  of  information  for  the  students.  Most  importantly  though,  the  teacher  as  facilitator 
should  be  genuine  and  strive  for  awareness  of  personal  attitudes.  The  teacher  also  needs  to  feel  acceptance  of  his  or 
her  own  feelings  thus  developing  an  authentic  relationship  with  the  students  (Rogers,  1961). 

Furthermore,  the  instructor  needs  to  be  self-actualized  in  order  to  foster  these  qualities  in  the  students 
(Patterson,  1973).  When  Rogers  and  Freiberg  (1994)  talked  to  students,  they  found  many  of  the  same  tasks  required 
to  be  become  a facilitator  were  also  wanted  by  students.  They  found  students  want  to  be  trusted  and  respected,  want 
freedom,  a place  where  people  care,  choices  to  make  decisions,  and  teachers  who  helped  them  succeed  (Rogers, 
1961).  In  a person-centered  educational  experience  not  only  will  the  learner  create  a creative  product  from  the 
learning  experience,  but  the  learner's  increased  self-actualization  can  also  be  considered  a creative  product.  As  a 
learner  becomes  more  self-actualized,  the  learner  will  be  able  to  perceive  reality  more  accurately;  accept  him  or 
herself  and  others;  understand  varying  views  and  perspectives;  become  more  spontaneous,  independent,  and  more 
creative  (Davis,  1992;  Maslow,  1970). 

Table  one  that  follows  describes  a model  of  person-centered  design  distilled  from  the  above  conceptual 
analysis  and  illustrates  how  creativity  operates  within  the  model.  Additionally,  this  person-centered  model  is 
contrasted  with  classic  instructionist  and  constructivist  approaches. 

A Person-Centered  Model  of  Instructional  Design 

In  this  section  we  move  on  to  our  second  focus  question  “how  can  Rogers’  theoretical  elements  be 
incorporated  into  an  instructional  design  model  that  will  be  useful  in  virtual,  web-based  environments?” 

Table  L Person-Centered  Instructional  Design  Model  to  the  ”4-Ps"  of  Creativity 


Person-Centered  Instruction 
(Rogers) 

"4-Ps"  of  Creativity 

Learner  Analysis 

Emphasis  is  on  the  learners'  interests,  personal  ability, 
and  prior  knowledge  of  a given  topic. 

Creative  Person 
Emphasis  on  the  person. 

Task  Formation 

Task  formation  proceeds  through  an  analysis  of 
integrating  students  knowledge  and  interests  around 
the  principles  of  the  content  or  discipline. 

Creative  Person 

Focus  on  developing  tasks  centered  around  the  person 

Learning  Environment  Selection 
Select  a non-competitive  environment  that  supports 
cooperative  learning  and  allows  the  learner  to  take 
responsible  control  over  it. 

Creative  Press 

Creating  a social/psychological  environment  to 
support  the  person,  process,  and  product 

Develop  Learning  Goal 

Develop  individual  achievable  objectives  within  the 
context  of  the  learning  experience  based  on  the 
students’  interests  and  abilities  and  contract  with  the 
instructor. 

Creative  Process 

Process  of  selecting  a goal  (creative  product) 

Individualized  Assessment  Development 
Work  with  students  to  develop  forms  of  self- 
evaluation. 

Creative  Process 

Develop  ways  to  test  goal  achievement 

Reciprocal  Teaching 

Organize  the  areas  of  interest  to  cover  in  the  topic  and 
sequence  in  a format  to  maximize  the  learning 
potential. 

Creative  Process 

Organizing  the  process  of  the  experience 
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Person-Centered  Instruction 
(Rogers) 

"4-Ps"  of  Creativity 

Selection  of  Instructional  Resources 
Identify  and  select  resources  to  enhance  the  learning 
experience  and  present  them  to  the  students.  The 
teacher  presents  himself  or  herself  as  a resource. 

Creative  Process  and  Press 

Selecting  resources  to  support  the  process  and  teacher 
taking  role  of  a resource  to  support  the 
social/psychological  environment 

Learner's  Self  Evaluation 

Learners  conduct  self-evaluation  based  on  the  contract 
of  the  level  of  personal  involvement,  self-initiated 
involvement,  and  pervasiveness,  which  shows  the 
significance  of  the  learning  experience. 

Creative  Process 

Testing  and  evaluating  the  process  and  the  creative 
product 

Outcomes  of  Process 

1.  Significant  Learning 

2.  Self-actualization 

3.  Creative  product 

(The  learner  will  show  not  only  an  accumulation  of 
knowledge  of  the  topic  but  also  satisfaction  in  the 
learning,  desire  to  master  the  experience,  and  a greater 
understanding  of  the  problem,  and  potential 
resolutions) 

Creative  Product 

Learning,  self-actualization,  and  a product 
emphasizing  the  originality  of  the  person  are  created. 

In  table  two  below  we  provide  a comparison  of  Rogers’  person-centered  elements  with  two  design  models  - 
- the  classic  instructional  design  of  Dick  & Carey  (1996)  and  constructivist  design  as  described  by  Jonassen  (1999). 

Table  2.  Comparison  of  Instructional  Design  Models 


Instructionalist  Design 
(Dick  and  Carey) 

Constructivist  Design 
(Jonassen) 

Person-Centered  Instruction 
(Rogers) 

Needs  Assessment 
Determine  what  is  the  optimal 
situation  and  the  actual  situation. 
Find  what  change  is  needed  to  fill 
the  gap  between  the  situations. 
This  will  identify  the  instructional 
goal. 

Problem  Definition 
Define  how  the  problem  is 
represented  and  the  manipulation 
space. 

Learner  Analvsis 
Emphasis  is  on  the  learners' 
interests,  personal  ability,  and 
prior  knowledge  of  a given  topic. 

Task  Analysis 

Determine  step  by  step  how  the 
students  will  accomplish  the 
goals. 

Determine  Problem  Dimensions 
Determine  what  is  needed  to  resolve 
the  problem. 

Task  Formation 

Task  formation  proceeds  through 
an  analysis  of  integrating  students 
knowledge  and  interests  around 
the  principles  of  the  content  or 
discipline. 

Learning  Environments 
No  focus  on  developing  a 
learning  environment. 

Describe  Learning  Environment 
Supports 

Determine  the  cases,  resources,  and 
tools  needed  to  provide  support  for 
the  learning  environment. 

Learning  Environment  Selection 
Select  a non-competitive 
environment  that  supports 
cooperative  learning  and  allows 
the  learner  to  take  responsible 
control  over  it. 

Performance  Objectives 
Development 

Write  performance  objectives  of 
what  students  will  be  able  to  do 
upon  completion  of  the 
instruction. 

Goals  and  Constraints  Are  Unstated 
Uncertainty  is  a plus.  Offer  no  rules 
for  predicting  the  outcome. 

Develop  Learning  Goal 
Develop  individual  achievable 
objectives  within  the  context  of 
the  learning  experience  based  on 
the  students'  interests  and  abilities 
and  contract  with  the  instructor. 

Assessment  Instrument 

Alternative  Assessments 

Individualized  Assessment 
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Instructionalist  Design 
(Dick  and  Carey) 

Constructivist  Design 
(Jonassen) 

Person-Centered  Instruction 
(Rogers) 

Development 

Develop  assessment  instruments 
to  measure  task  achievement. 

Development 

Provide_opportunities  for  flexible, 
creative  demonstrations  of  student 
understanding. 

Development 

Work  with  students  to  develop 
forms  of  self-evaluation. 

Instructional  Strategy 
Sequence  and  organize  the 
information  as  an  instructional 
strategy  for  delivery. 

Instructional  Strategy 
Coaching,  modeling,  and  scaffolding 
support  and  challenge  the  learner  to 
succeed. 

Reciprocal  Teaching 
Organize  the  areas  of  interest  to 
cover  in  the  topic  and  sequence  in 
a format  to  maximize  the  learning 
potential. 

Selection  of  Instructional 
Resources 

Develop  and  select  instructional 
resources. 

Provide  the  Problem  Manipulation 
Space 

Include  objects  and  tools  that  are 
required  for  the  learner  to  manipulate 
the  environment. 

Selection  of  Instructional 
Resources 

Identify  and  select  resources  to 
enhance  the  learning  experience 
and  present  them  to  the  students. 
The  teacher  presents  himself  or 
herself  as  a resource. 

Evaluation  of  Learning 
Design  and  conduct  a formative 
evaluation  to  determine  the 
effectiveness  of  the  instruction. 
Conduct  a summative  evaluation 
to  verify  the  effectiveness  of  the 
instructional  event. 

Evaluation  of  Learning 
Evaluate  the  problem  solving  process 
and  viability  of  the  solution. 

Learner's  Self  Evaluation 
Learners  conduct  self-evaluation 
based  on  the  contract  of  the  level 
of  personal  involvement,  self- 
initiated  involvement,  and 
pervasiveness,  which  shows  the 
significance  of  the  learning 
experience. 

Outcome  of  the  Process 
Based  on  assessment  scores, 
formative  evaluation,  and 
summative  evaluation 
achievement  of  the  goal  can  be 
determined. 

Outcome  of  the  Process 
Outcomes  are  understanding  and 
further  inquiry. 

Outcomes  of  Process 

1 . Significant  Learning 

2.  Self-actualization 

3.  Creative  product 

(The  learner  will  show  not  only 
an  accumulation  of  knowledge  of 
the  topic  as  well  as  a creative 
product  but  also  satisfaction  in  the 
learning,  desire  to  master  the 
experience,  and  a greater 
understanding  of  the  problem,  and 
potential  resolutions) 

Conclusions 

What  are  the  potential  benefits  and  drawbacks  of  using  a person-centered  model  of  instructional  design? 
By  emphasizing  students’  interests  and  abilities,  courses  taught  in  virtual  environments  such  as  certain  applications 
delivered  via  the  WWW,  can  create  an  atmosphere  of  mutual  participation  and  allow  for  accommodations  of  various 
skill  and  ability  levels.  Students  can  exercise  the  freedom  to  choose,  which  is  encouraged  by  the  user-controlled 
hypermedia  web  environment.  By  utilizing  the  person-centered  approach  in  design  of  instruction,  students  can  take 
full  advantage  of  the  very  features  of  virtual  environments  that  are  thought  to  promote  engagement  and  enhance 
learning.  In  other  words,  the  design  of  instruction  using  person-centered  design  is  a good  fit  with  a user-controlled, 
web-based  instructional  environment  such  as  the  one  discussed  in  the  case  study  above. 

However,  the  elements  promoting  the  success  of  such  an  approach— user  responsibility,  ability  to  be  self- 
assessing  and  proactive  in  learning—  are  the  very  elements,  when  lacking,  which  will  result  in  an  instructional 
experience  that  is  non-productive  at  best  and  frustrating  at  worst.  For  example,  how  will  students  who  have 
incorrectly  assessed  their  abilities  fare  in  such  a free  choice  environment?  Indeed,  what  kinds  of  self-assessment 
tools  will  need  to  be  available  for  students  (and  instructors)  to  make  such  appraisals  of  skill  and  knowledge?  What 
opportunities  or  interim  assistance  is  needed  to  aid  these  students  in  fully  participating  in  the  course  and  learning  the 
material?  Will  students  who  often  expect  to  obtain  course  material  via  lecture  and  didactic  instruction  feel  cheated 
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if  the  instructor  relies  on  them  to  shape  their  own  course  experiences?  Will  these  students  have  a point?  What  is  the 
proper  role  of  the  instructor? 

While  the  caveats  for  using  a person-centered  model  are  valid,  we  posit  from  our  initial  exploration  of  the 
utility  of  adapting  central  concepts  and  principles  from  Rogers’  person-centered  approach  has  shown  it  to  be 
potentially  useful.  Designing  instruction  for  virtual,  learner-centered  courses  or  learning  experiences  requires  an 
approach  focused  on  the  learner.  The  use  of  such  approaches  may  ameliorate  the  issues  raised  by  the  Fordist  debate 
and  ensure  that  on-line  courses  and  virtual  communication  and  collaboration  environments  develop  in  ways  that 
truly  exploit  the  instructional  potential  to  develop  self-actualized,  independent  learner. 
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Abstract 

This  paper  presents  strategies  and  rationales  for  implementing  certain  instructional  techniques  to 
move  a class  from  cohort  to  community . The  context  is  the  new  Distance  Master's  program  in 

Instructional  Systems  Technology  at  Indiana  University.  The  authors  give  suggestions  for  instructional 
and  non-ins  tructional  strategies  that  have  students  interacting  at  the  levels  of  discussion,  cooperation  and 
collaboration.  These  strategies  are  cross-indicated  with  their  intended  outcomes,  that  is,  strengthening 
the  feeling  of  community  as  defined  by  a set  of  characteristics,  which  are  adapted  from  Schwier  (in 
press).  Suggestions  for  evaluation  techniques  are  also  presented,  as  are  questions  for  further  research. 

Introduction 

The  shift  from  traditional  classroom  education  to  computer-mediated  distance  learning  poses  enormous 
challenges  to  instructors  and  learners.  The  concept  of  the  classroom  where  students  meet  to  interact  with  other 
learners  and  the  instructor  no  longer  exists.  The  instructor  can  no  longer  “look”  around  the  room  to  see  if  students 
are  attending  to  the  material,  bored  or  confused.  Learners  lack  a natural  social  outlet  to  engage  with  other  learners 
thus  leading  to  feelings  of  isolation.  The  learner  is  now  engaged  with  the  computer  instead  of  other  learners.  The  big 
question  for  our  project  is  “How  do  we  structure  the  course  design  so  learners  have  mechanisms  to  connect  with 
each  other  and  form  community.”  How  do  we  overcome  the  characteristics  of  the  medium  so  that  learners  feel 
connected  to  the  instructor  and  other  learners? 

The  literature  on  effective  teaching  and  learning  promotes  several  “big  ideas”  that  we  used  as  foundations 
for  our  recommendations.  These  include  Vygotsky’s  (1978)  social  development  theory  and  the  Seven  Principles  for 
Good  Practice  in  Undergraduate  Education  (Chickering  & Gamson,  1987).  Vygotsky’s  social  development  theory 
states  that  social  interaction  is  vital  to  cognitive  development;  all  higher-order  functions  originate  as  the 
relationships  among  individuals.  To  scaffold  learning  we  must  require  learners  to  interact  with  the  content,  the 
teacher  and  each  other.  Our  strategies  focus  on  promoting  communication,  social  interaction  and  participation.  Many 
of  the  principles,  theories  and  strategies  we  encountered  reflect  the  Seven  Principles  of  Good  Practice  in 
Undergraduate  education  (Chickering  & Gamson,  1987).  At  their  core,  each  of  the  seven  principles  focuses  on 
interaction.  In  1996,  Moore  and  Kearsley  described  three  types  of  interactions  that  are  necessary  in  distance 
education:  learner  to  learner,  leaner  to  content  and  learner  to  instructor.  We  would  argue  that  these  three  types  of 
interaction  are  necessary  in  education  regardless  of  where  or  how  it  takes  place. 

Characteristics  of  Community 

There  is  much  discussion  of  learning  communities,  communities  of  practices,  and  virtual  or  online  (social) 
communities.  Although  each  type  of  community  has  its  distinct  characteristics  and  requirements,  there  are  many 
things  they  share  in  common.  What  we  are  endeavoring  to  create  will  be  a combination  of  all  of  the  aforementioned 
communities:  a community  of  practice  (since  our  cohort  will  be  from  the  same  company)  that  is  involved  in  mutual 
learning  online.  Because  of  these  special  characteristics,  some  things  do  not  apply.  For  example,  there  is  much  talk 
in  the  virtual  community  literature  about  attracting  members  and  defining  the  community  based  on  common 
interests.  In  our  case,  this  cohort  is  thrown  together  and  “forced”  to  form  community.  Outside  members  are  not 
encouraged  to  participate,  mainly  because  the  common  interest  in  this  case  is  “taking  the  Distance  Masters  in  1ST 
from  IUB.”  In  a terrestrial  community  of  practice,  members  might  see  each  other  at  work,  or  meet  in  person  once  a 
week  to  deal  with  issues  in  their  work  lives.  This  will  not  exactly  be  the  case  for  our  community;  although  they  will 
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probably  have  some  work  issues  in  common,  they  are  not  a group  of  “teachers”  or  “nurses”  or  “engineers”  who 
share  vast  amounts  of  experience  and  knowledge.  Unlike  an  informal  learning  community,  which  spins  itself  from 
nothing  and  is  based  on  a variety  of  people  coming  together  for  informal  learning  purposes  and  where  the  direction 
of  both  the  learning  and  the  community  is  malleable,  our  learning  community  will  exist  within  strict  parameters  of 
this  coursework.  Obviously,  members  will  be  encouraged  to  bring  other  experiences  and  knowledge  to  bear  on  their 
coursework,  but  at  the  end  of  the  day,  the  learning  in  question  will  be  much  more  restricted  than  an  informal 
learning  community. 

Selznik  (1996)  identifies  seven  elements  of  community:  history,  identity,  mutuality,  plurality,  autonomy, 
participation,  and  integration . With  respect  to  virtual  learning  communities,  Schwier  (in  press)  adds:  an  orientation 
to  the  future,  technology,  and  learning.  Some  of  these  characteristics  of  community  will  be  present  from  the 
beginning.  Others,  the  cohort  will  have  to  grow  into.  We  will  describe  the  features  of  these  10  characteristics,  and 
discuss  how  we  will  use  them  for  our  purposes. 

Selznik  notes  that  communities  are  stronger  when  their  members  share  history  and  culture,  rather  than 
simply  abstract  general  interests.  Unlike  an  established  terrestrial  community,  the  nascent  community  forming  from 
the  distance  education  masters  program  will  not  have  a shared  history.  Their  history,  like  their  identity,  will  have  to 
grow  and  develop  through  their  interactions  with  each  other. 

We  believe  that  a shared  sense  of  identity  will  develop  in  this  cohort,  and  will  strengthen  their  communal 
identity.  Schwier’ s suggested  strategies  for  fostering  identity  include  team-building  exercises,  developing 
community  logos,  and  public  acknowledgement  of  individual  and  group  accomplishments  within  the  community.  He 
also  notes  the  importance  of  articulating  the  “focus  or  purpose  of  the  community”  and  outlining  the  requirements 
and  rituals.  The  structure  of  the  courses  allows  for  frequent  and  obvious  reiteration  of  community  focus,  and  events 
such  as  orientation  can  help  the  group  define  its  own  rituals  and  norms. 

The  very  fundamentals  of  a learning  community  require  interdependence  and  reciprocity,  what  Selznik 
terms  mutuality.  Since  our  focus  is  on  cooperative  and  collaborative  learning,  this  mutuality  will  develop  naturally. 
Schwier  also  recommends  asking  “leading  questions  that  encourage  members  of  the  community  to  invest  in 
concerns  held  by  other  members,  and  to  share  ideas  and  possible  solutions”  (p.  5).  This  type  of  interaction  can  be 
encouraged  at  course-level  in  the  class  forum,  and  on  a social  level  in  the  Online  Cafe. 

We  combined  Selznik’s  terms  history,  mutuality,  and  identity  into  a larger  category  called  group  identity. 
By  combining  these  three  concepts  we  emphasize  the  fundamental  importance  of  group  identity  in  fostering 
community.  Although  one  of  our  goals  in  the  next  few  semesters  is  to  help  students  begin  to  construct  a history 
relevant  to  their  community,  this  is  not  something  that  can  be  imposed  upon  the  group  from  outside.  It  has  to  grow 
from  the  sharing  of  each  individual’s  history  and  the  links  that  the  learners  form  with  each  other  based  on  their 
experiences.  These  links  are  characterized  by  interdependence  and  reciprocity,  in  other  words  mutuality.  Group 
identity  results  from  this  history  and  mutuality,  and  from  making  the  budding  community  history  public  and 
available  to  all,  especially  newcomers. 

Plurality \ according  to  Selznik,  results  when  many  different  types  of  interactions  amongst  members  of  a 
community  occur,  often  rooted  by  individuals’  membership  in  other  communities  (work,  neighborhood,  church,  etc.) 
that  intersect.  We  replaced  plurality  with  social  interaction.  Given  a virtual  community,  one  that  to  some  extent  is 
externally  imposed,  the  opportunities  for  plurality  are  limited  compared  to  those  available  to  geographic 
communities.  By  providing  opportunity  for  and  the  expectation  of  social  interaction  among  participants,  we  purport 
the  program  will  provide  the  plurality  needed. 

Autonomy  of  individual  members  within  the  community,  especially  within  an  academic  setting,  is  important 
to  foster.  We  will  encourage  thoughtful,  personal  postings  within  the  forum,  to  avoid  group-think  and  “me  too,  I 
agree”  contributions.  Students  will  receive  basic  instruction  on  netiquette  and  will  be  encouraged  to  continually 
address  evolving  group  norms  to  maintain  respectful  communication  and  to  build  consensus.  We  use  individual 
identity  in  place  of  autonomy  to  underscore  the  importance  of  both  group  and  individual  identities  within  a virtual 
learning  environment. 

In  the  case  of  a virtual  community,  participation , both  social  and  academic,  is  integral.  Without  active 
participation  in  discussions  and  other  class  activities,  the  learner  is  not  part  of  the  community;  indeed,  the  learner 
does  not  even  “exist.”  This  is  one  core  distinction  between  being  a passive  member  of  a physical  community  where 
you  are  seen  and  your  presence  is  noted  and  registered  in  the  minds  of  others.  In  a virtual  community,  you  must 
make  a concerted  effort  to  communicate  with  others  in  order  to  exist.  At  the  same  time,  allowances  must  be  made 
for  learners  to  shape  the  participation,  both  in  structure  (number/kind  of  postings)  and  in  content  (managing  the 
discussion  of  subjects  interesting  to  them). 

The  future  orientation  of  a learning  community  can  operate  at  a number  of  different  levels.  A stronger 
community  bond  will  be  formed  when  a particular  cohort  goes  through  a number  of  courses  together,  moving 
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toward  their  finishing  the  program  and  earning  a degree.  It  can  be  argued  that  a learning  community  can  develop 
within  the  constraints  of  a single  four-month  course,  but  it  is  much  more  likely  that  students  will  form  long-lasting 
academic  and  social  bonds  throughout  an  entire  program.  Visioning  exercises  and  direction  of  learning  activities 
(having  participants  describe  how  what  they  learned  will  help  them  in  future  learning  and  in  their  work)  can  also 
give  the  community  a focus  on  the  future.  In  our  case,  the  community's  view  of  the  future  may  be  limited  to  the  two 
or  three  years  they  spend  in  the  program.  However,  it  is  possible  that  they  will  continue  to  maintain  community  ties 
once  they  have  earned  they  degrees  and  are  working  again.  It  is  also  possible  that  members  of  the  Fall  2000 
community  would  end  up  wanting  to  remain  part  of  the  Distance  Masters  community  after  they  graduate,  and  would 
like  to  integrate  themselves  with  the  new  incoming  cohorts.  This  may  pose  particular  problems  of  negotiation  and 
fit;  is  there  a role  for  graduated  members  to  “return”  virtually  and  engage  with  students  working  through  the 
program? 

Schwier  notes  that  “the  nature  of  the  learning  can  be  broadly  defined  and  contextual’ ’(p.  4)  but  is  a 
necessary  part  of  a virtual  learning  community.  For  our  purposes,  the  learning  involved  is  more  specific  and 
structured;  the  cohort  moves  through  a set  of  core  courses  together,  in  a particular  order.  Our  goal  is  to  foster 
community  among  them  before  they  finish  the  first  year,  so  that  although  they  will  go  on  to  take  other  courses  with 
other  distance  learners,  they  will  not  only  maintain  ties  with  their  initial  cohort  community,  but  will  also  have 
learned  the  foundations  of  virtual  community  creation  and  will  use  these  skills  in  other  classes.  We  have  changed 
Schwier’ s term  learning  to  knowledge  generation . 

According  to  Schwier,  “communities  are  built  or  dismantled  by  those  in  the  communities,  not  by  the  people 
organizing  or  managing  them”  (p.  2).  As  they  mature,  communities  define  their  own  social  rules  of  conduct  and 
select  their  own  leaders,  assuming  ownership  of  their  governance  and  norms.  Learning  communities,  note  Palloff 
and  Pratt  (1999),  exhibit  evidence  of  socially  constructed  meaning,  willingness  to  critically  evaluate  the  work  of 
others,  again  assuming  ownership  of  their  knowledge  creation  and  sharing. 

Integration  of  all  of  these  elements  is  necessary  for  a strong  community.  Schwier  suggests  creating  belief 
statements  and  evolving  group  norms,  and  adhering  to  a learner-centered  philosophy  that  “supports  individual 
expression  while  building  a group  identity”  (p.  5). 

Finally,  technology  is  an  important  consideration  for  us:  although  it  is  thanks  to  certain  technologies  that 
virtual  community-building  is  even  possible,  there  are  certain  limitations  put  upon  the  group  because  of  technology. 
Although  it  is  the  conduit  for  discourse,  it  can  also  exclude  or  discourage  people.  Tools  that  are  complicated, 
unavailable  for  a certain  platform,  that  are  slow  and  cumbersome  can  all  render  the  discussion  process  less  than 
ideal,  and  members  who  do  not  actively  participate  essentially  leave  the  community.  Although  Schwier  recommends 
using  technology  compatible  with  older,  less  costly  equipment  to  render  the  community  more  inclusive,  this  is  not  a 
concern  for  us. 

Based  on  Selznik’s  (1996)  seven  characteristics  and  Schwier’s  (in  press)  additional  three  characteristics  of 
community,  we  have  assembled  the  aforementioned  six  key  elements  of  community.  From  these  elements,  we  define 
community  as:  a group  of  people  who  are  brought  together  to  share  and  generate  knowledge  in  a mutually 
supportive  and  reciprocal  manner.  Its  characteristics  are  ownership,  social  interaction,  group  identity,  individual 
identity,  participation,  and  knowledge  generation.  Furthermore,  integration  of  all  of  these  elements  is  necessary 
for  a strong  community. 

Having  defined  some  of  the  particular  characteristics  of  a virtual  community,  we  will  now  turn  to  some 
basic  strategies  for  creating  community,  Palloff  and  Pratt  (1999)  recommend  these  steps: 

• Clearly  define  the  purpose  of  the  group 

• Create  a distinctive  gathering  place  for  the  group 

• Promote  effective  leadership  from  within. 

• Define  norms  and  a clear  code  of  conduct, 

• Allow  for  a range  of  member  roles. 

• Allow  for  and  facilitate  subgroups. 

• Allow  members  to  resolve  their  own  disputes  (p.  24) 

In  our  case,  many  of  these  steps  are  automatic,  but  they  should  still  be  given  careful  consideration.  For 
example,  the  general  purpose  of  the  community  is  defined  as  “the  Fall  2000  cohort  for  the  1ST  Distance  Masters 
program.”  However,  instructors  or  organizers  may  have  more  specific  goals  and  purposes  from  the  beginning,  and 
even  if  they  do  not,  other  purposes  may  emerge  from  the  community  throughout  the  term.  Pallof  and  Pratt  (1999), 
surprisingly,  do  not  put  much  emphasis  on  the  communicative  aspect  of  community  without  which  a virtual  learning 
community  cannot  exist. 
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We  feel  that  one  of  the  most  important  indicators  of  a learning  community  is  the  first:  when  students 
communicate  not  only  on  an  academic  level  but  on  a personal  level.  Working  together  towards  the  goals  of  the 
course  is  what  they  are  “supposed”  to  be  doing.  When  they  begin  to  talk  about  their  personal  lives  (families, 
hobbies,  jobs),  their  triumphs  and  trials  with  being  a distance  student  (scheduling,  technical  problems,  disagreement 
with  pedagogy),  when  they  seek  each  other’s  counsel  for  other  areas  of  their  life  (job  change,  which  elective  course 
to  take  next,  family  issues),  this  is  the  point  at  which  we  feel  they  are  comfortable  as  a community.  There  is  a good 
chance  that  not  everyone  will  be  everyone  else’s  best  friend.  However,  when  a majority  of  the  members  feel  they  are 
in  a safe  enough  space  to  “speak  up”  about  things  in  the  public  forum,  rather  than  in  individual  e-mail  messages, 
then  this  is  evidence  of  a successful  community.  There  may  be  a few  members  of  the  community  who  do  not  feel 
that  the  Online  Cafe  is  an  appropriate  place  to  discuss  non-academic  subjects,  and  it  is  the  role  of  the  mentor  and  the 
community  members  to  make  the  Cafe  a welcoming  place  for  this  type  of  discussion.  As  in  every  type  of 
community,  there  will  be  some  people  who  opt  out  of  certain  discussions,  or  even  out  of  all  “non-official” 
discussion,  but  this  is  quite  normal.  There  will  probably  be  smaller  communities  within  the  larger  online  class, 
people  who  form  bonds  and  discuss  the  course  work  and  their  lives,  but  not  on  the  general  forum.  These  differences 
can  appear  for  a variety  of  reasons;  Eastmond  (1995)  found  divisions  on  age,  gender,  experience,  and  learning  style 
lines.  However,  he  also  found  that  the  groups  often  transcended  age  and  gender,  for  example,  two  characteristics  that 
might,  in  a traditional  classroom,  be  impediments. 

The  final  step  in  creation  of  an  online  community  is  to  evaluate  whether  a community  has  formed,  and  if 
so,  in  what  ways  has  the  community  aspect  contributed  to  learning.  Our  project  will  address  methods  for  performing 
the  first  evaluation  of  whether  community  has  formed. 

Definitions 

We  will  examine  ways  to  use  certain  instructional  strategies  to  work  to  move  the  cohort  toward  a 
community.  We  suggest  encouraging  interaction  at  three  levels:  discussion,  cooperation,  and  collaboration. 

Cohort 

The  cohort  is  the  group  of  students  going  through  the  core  classes  as  a group.  They  may  have  an  initial 
connection,  such  as  a common  employer,  but  it  does  not  necessarily  constitute  a strong  bond. 

Discussion 

Discussion  is  the  basic  means  of  communication  in  an  online  format.  Students  must  participate  in 
discussion  to  have  any  sort  of  presence  in  the  class  whatsoever.  Discussion  can  be  focused  around  readings,  lectures, 
and  any  other  ideas  based  on  course  content  or  course  administration.  Discussion  can  occur  asynchronously  in  the 
SSF  or  via  e-mail,  or  synchronously  via  chat  rooms  or  telephone. 

Cooperation 

Cooperation  entails  students  working  in  groups  or  otherwise  dividing  up  tasks.  A machine  metaphor  can 
illustrate  cooperation  in  the  classroom:  different  parts  of  the  machine  perform  different  functions  and  goals,  but 
work  together  towards  a similar  end.  For  example,  students  may  divide  up  a project,  but  are  eventually  assigned 
individual  grades  for  their  work.  Examples  of  cooperative  tasks  include:  dividing  up  sections  of  a report  to  write  and 
doing  peer  review  of  each  other’s  work. 

Collaboration 

Collaboration  is  the  most  integrated  form  of  group  work,  and  is  therefore  potentially  the  most  difficult  and 
the  most  rewarding.  In  the  case  of  collaboration,  the  group  members  work  toward  a common  goal,  one  that  carries  a 
mutual  investment.  For  example,  students  may  each  work  on  every  part  of  the  report,  consulting  each  other  and  re- 
reading each  other’s  edits.  They  are  invested  in  every  part  of  the  project  because  they  will  share  a common  grade. 
Examples  of  collaborative  tasks  include  group  writing  and  creating  an  ID  model. 


Community 

A virtual  learning  community,  as  described  in  the  introduction,  is  one  of  the  ultimate  goals  of  the  core  courses. 
The  three  levels  of  interactions  can  be  compared  by  several  characteristics,  as  in  the  table  below. 


Discussion 

Cooperation 

Collaboration 

Learning 

Information 

transmission 

Knowledge 

transmission 

Knowledge 

generation 

Inquiry 

Individual  inquiry 

Delegation  of  tasks 

Common  inquiry 

Decision-making 

Agree  to  disagree 

Vote  (majority  rules) 

Social  negotiation  to 
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consensus 

Goals/agendas 

Multiple  goals/ 
multiple  agendas 

One  goal/  multiple 
agendas 

One  goal/  one 
agenda 

Accountability 

Individual 

accountability 

Individual 

accountability 

Group 

accountability 

Learning 
relationship  g. 

Complete 

independence 

Partial 

interdependence 

Complete 

interdependence 

Description  of  1ST  Core 

The  term  “Core”  is  used  in  the  1ST  department  to  denote  four  courses  that  all  graduate  students  take  in  their 
first  year  in  the  program.  Traditionally  R51 1 (2  credits),  R52 1/522  (4  credits),  R580  (1  credit)  are  offered  in  the  Fall 
term;  R561  (3  credits)  is  offered  in  the  Spring.  It  is  usually  the  case  that  the  new  students  (both  Masters  and 
Doctoral)  take  these  classes  as  a group;  they  form  a cohort  that  goes  through  at  least  the  first  year  of  courses 
together.  The  cohort  identity  is  important  to  the  1ST  program,  and  it  is  something  that  will  be  actively  cultivated  in 
the  online  Masters  program.  Tangential  to  the  cohort  identity  is  the  community-building  that  is  undertaken  to 
integrate  new  students  into  the  1ST  program.  The  social  aspect  of  the  community  is  nurtured  through  happy  hours, 
the  1ST  picnic  in  the  fall  and  the  Follies  show  in  the  spring,  and  informal  pairing  new  students  with  old  ones. 
Academically,  the  1ST  community  is  built  through  the  identity  of  the  Rookie  cohort,  through  the  rookies  taking  non- 
Core  classes  (R547,  Y520,  etc.)  with  upper  year  students,  through  rookie  interaction  with  upper  year  AIs  in  Core 
and  non-Core  classes,  etc.  The  1ST  department  is  also  very  much  linked  to  its  alumni,  through  alumni  presentations 
in  R580  (Grads  with  Gigs)  and  networking  at  conferences. 

The  pedagogy  is  rooted  in  project-based  learning  and  team-based  work.  Much  of  the  learning  is  hands-on, 
and  students  often  work  with  real-world  clients.  There  is  a focus  on  an  integrated  curriculum  and  many  of  the 
courses  are  team-taught.  The  different  research  areas  of  the  faculty  (for  e.g.,  corporate  vs.  higher  education  vs.  K-12) 
expose  all  students  to  multiple  academic  perspectives.  The  international  nature  of  the  program  (approximately  one- 
third  of  the  students  are  non-U.  S.  citizens)  exposes  all  students  to  different  ways  of  learning  and  working.  Because 
of  the  content,  there  is  an  emphasis  on  technological  competence,  although  the  skill  levels  of  both  entering  and 
graduating  students  vary  immensely.  Although  the  use  of  technology  in  education  is  important  to  1ST,  technology  is 
a means,  not  an  end,  and  its  use  is  firmly  rooted  in  pedagogy. 

The  associations  that  1ST  has  with  other  departments,  including  Educational  Psychology,  Language 
Education,  the  Kelly  School  of  Business,  the  School  of  Library  Science,  etc.,  contribute  to  an  integrated  and 
interdisciplinary  academic  environment.  Most  of  these  departments  offer  online  courses  that  can  be  used  by 
Distance  Masters  students  as  electives. 


Core  Instructional  Strategies  and  Rationales 


Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

j 

ip 

4 

w 

A 

fiP 

A 

fi? 

Students  participate  in  a face-to- 
face  orientation  on  campus. 

Face-to-face  interactions  allow  to  people  to 
create  strong  initial  bonds,  which  will  lead  to 
a greater  sense  of  community  right  from  the 
beginning. 

i 

i 

i 

W 

i 

fiP 

Students  will  learn  about  online 
communication,  including  rules 
of  netiquette 

Online  communication  is  vastly  different  from 
more  traditional  forms  of  communications 
(Black,  1995). 
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Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

i 

fiF 

A 

Sr 

A 

Sr 

Students  will  undergo  training  in 
using  SiteScape  Forum,  e-mail, 
majordomo  creation,  basic  web 
searches,  and  MS  Word  for 
collaborative  writing  purposes. 

To  help  reduce  barriers  to  effective  learning 
and  establishing  social  relationships, 
participants  should  be  given  the  opportunity  to 
build  confidence  and  competence  with  the 
distance  education  process  and  supporting 
technologies  (IDE,  2.2). 

* 

i 

SF 

A 

Sr 

Students  will  post  photos  of  in 
SiteScape  Forum  at  Orientation. 

Connecting  people’s  names  and  faces  is  a first 
big  step  to  forming  bonds. 

i 

i 

W 

A 

fir 

A 

fir 

i 

fir 

Students  will  participate  in  a 
content-based  group  project  that 
requires  that  they  negotiate  the 
exact  content. 

People  will  form  strong  personal  and 
academic  bonds  through  shared  adversity 
(Ruhleder,  1999). 

A 

w 

i 

fir 

Students  will  be  required  to  eat 
lunch  as  a group  two  days  during 
Orientation. 

People  who  have  a social  connection  to  the 
group  will  work  better  together  (Palloff  & 
Pratt,  1999). 

A 

Sr 

A 

fir 

Students  will  be  given  the 
opportunity  to  participate  in  at 
least  two  evening  social  activities. 

People  who  have  a social  connection  to  the 
group  will  work  better  together  (Palloff  & 
Pratt,  1999) 

* 

Sr 

First  posting  should  be  a non- 
graded/non-credited  assignment 
(e.g.  biography). 

Students  need  non-threatening,  interesting 
ways  to  begin  creating  online  community 
(Funaro,  1999). 

A 

w 

A 

fiF 

a 

i 

Sr 

i 

fir 

i 

fir 

Create  an  online  cafe  that  will 
serve  as  a non-course-specific 
conversation  area  to  encourage 
off-task  communication. 

People  need  distinctions  between  work  and 
play  (Palloff  & Pratt,  1 999). 

a 

A 

Sr 

i 

Sr 

i 

fir 

Encourage  instructor  and  distance 
mentors  to  participate  in  social 
interactions,  especially  in  the 
early  stages  of  the  course. 

Social  interactions  between  and  among 
learners  enrich  the  learning  community  and 
should  be  supported  in  the  instructional  design 
of  the  course  (IDE,  2.5). 

A 

fir 

i 

fir 

A 

fir 

Students  will  be  encouraged  to 
share,  in  the  online  cafe, 
information  about  their  non- 
academic  lives,  for  example, 
offering  mutual  support  in  term  of 
how  they  are  keeping  up  with 
their  job  and  school  schedule. 
Students  should  be  encouraged  to 
offer  successful  strategies  to  the 
class. 

Reciprocity  and  help  are  two  important 
hallmarks  of  community.  Students  who  take 
an  interest  in  each  other’s  well  being,  both 
academic  and  social,  will  have  more  of  a 
support  system  of  peers  than  those  who  do  not 
(Wellman  & Gulia,  1999). 

R511  Section 

Description  of  R511  (from  course  syllabus) 

R511,  Instructional  Technology  Foundations  I,  is  a two-credit  course  that  has  historically  been  offered  each  fall 
semester.  This  course  is  required  by  all  1ST  Masters  students  and  is  typically  taken  concurrently  with  R52 1/522, 
Instructional  Design  and  Development,  and  R580,  1ST  Colloquium.  It  is  team-taught  by  two  faculty  members  and 
one  graduate  assistant  who  has  taken  the  course. 

The  overall  objective  of  this  course  is  to  provide  a comprehensive  introduction  to  the  field  and  profession 
of  Instructional  Technology  (IT).  Since  most  entering  1ST  students  come  from  fields  other  than  instructional 
technology,  R511  gives  newcomers  a sense  of  history  and  an  explanation  of  how  the  components  of  the  field  fit 
together.  There  is  a particular  emphasis  on  the  evolution  of  the  “big  ideas”  of  the  field. 
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In  the  onsite  version  of  R511,  class  meetings  occur  once  per  week  in  2-hour  sessions.  Directed  readings 
compiled  in  a course  packet  are  provided  as  practical  resources  to  support  assignments  and  class  discussion 
activities  in  the  course.  Most  class  periods  are  divided  into  two  portions: 
1)  During  the  first  hour,  each  of  the  three  instructors  facilitates  a group  discussion  among  15-20  students  about 
assigned  readings.  2)  The  remaining  portion  of  the  class  time  is  devoted  to  further  lecture  and  clarification  about 
topics  contained  in  the  readings. 

Students  are  graded  according  to  participation  in  class  discussion,  personal  synthesis  and  reflection  (as 
noted  in  weekly  minute-papers  collected  at  the  end  of  each  class),  three  individual  written  essays  (one  team-based, 
two  individual),  and  a final  exam  or  written  essay. 

R511  Instructional  Strategies  and  Rationales 


Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

i 

SF 

A fundamental  element  for 
success  for  the  distance  students 
is  an  understanding  of  the  key 
expectations 

•how  much  time  the  course  will 
require 

•the  level  of  performance  that  is 
expected  of  them 

•the  demands  that  participating  in  the 
core  will  have  on  their  time. 

Students,  but  especially  students 
learning  at  a distance,  need  to  have 
expectations,  assumptions, 
deadlines,  etc.,  made  explicit  and 
kept  clear  (Palloff  & Pratt,  1999). 

Understanding  and  respecting 
expecta-tions  for  participation  and 
performance  will  be  critical  to  the 
students’  success.  Taking  Core 
online  will  be  more  demanding 
than  doing  it  face-to-face. 

i 

SF 

j 

i 

SF 

1 

F 

Instructors  will  assign 
discussion  roles  (facilitator, 
summarizer,  devil’s  advocate, 
etc.)  to  encourage  shy  members 
and  force  students  to  think  in 
different  ways  about  the 
material  and  about  the 
discussion  of  the  material. 

Students  should  be  challenged  to 
engage  the  material  from  different 
perspectives;  different  roles 
improve  leamer-leamer  interaction 
and  improve  learner-material 
interaction. 

J 

sF 

J 

SF 

J 

SF 

Students  will  be  expected  to 
take  part  in  regular  peer  reviews 
by  critically  evaluating  each 
other’s  papers. 

It  is  important  to  develop  a critical 
eye  towards  other  community 
members’  work. 

j 

sF 

j 

SF 

J 

SF 

Each  week,  someone  from  each 
group  will  summarize  their 
group’s  discussion  and  post  the 
results  for  the  other  groups  to 
read. 

Bringing  from  small  groups  to  the 
larger  group  provides  for  more 
viewpoints  and  better  discussion. 

J 

SF 

j 

SF 

J 

SF 

Students  will  be  divided  into  3- 
4 small  groups  for  discussion  of 

Small  groups  facilitate  better 
discussion  (Hiltz,  1998)  for 
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Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

readings  and  course  projects. 

learner-material  interaction. 

i 

w 

4 

A 

SF 

Students  will  fill  out  weekly  “1- 
minute  evaluation”  web  form, 
to  instructor  only.  Possible 
topics  include  what  you 
liked/disliked  about  the  week’s 
work,  how  you  can  transfer  this 
knowledge  to  your  work,  and 
generally  how  you  are  feeling. 

To  better  assimilate  and  process 
what  they  have  learned,  students 
require  a forum  to  critically  reflect 
on  the  material  and  on  themselves 
as  learners  (Palloff  & Pratt,  1999). 
Keeping  in  touch  with  the 
professor  improves  learner-faculty 
interaction. 

i 

W 

i 

w 

A 

SP 

Instructors  will  require  high- 
quality  online  interactions  with 
peers  and  discussions  of 
readings  by  making  a portion  of 
the  grade  dependent  on  it.  (We 
recommend  at  least  25%). 

Effective  learning  environments 
should  provide  frequent  and 
meaningful  interactions  among 
learners.  (IDE,  2.1) 

Good  practice  encourages 
cooperation  among  students 
(Chickering  & Gamson,  1987). 

i 

w 

i 

W 

A 

w 

Instructor  and/or  mentor  will 
model  ways  to  produce  lively, 
constructive  discussion: 
questions  should  be  open- 
ended,  but  focused  on  students5 
interpretation  of  the  text. 

One  of  the  best  ways  to  keep 
discussion  on  topic  and  students 
motivated  is  to  participate  actively 
in  the  conversation  (Beaudin 
1999). 

A 

W 

i 

w 

A 

fip 

Instructor  will  point  out 
excellent  discussion,  postings, 
interactions,  etc.  of  other 
students  to  continually  promote 
high  expectations  and  model 
good  interaction. 

Good  practice  encourages  prompt 
feedback  (Chickering  & Gamson, 
1987).  Faculty-learner  interaction 
improved  by  attentive  professor. 

i 

w 

J 

w 

a 

fip 

As  needed,  instructor  will 
revisit  netiquette  and  general 
interaction  issues,  and  stresses 
the  importance  of  interacting  in 
a respectful  way.  Have  the 
community  develop  group 
norms  based  on  emergent 
issues. 

Social  negotiation  leads  to  the 
creation  of  a safe  space,  which  is 
essential  for  learning  (Palloff  & 
Pratt,  1999). 

i 

Br 

Students  will  be  expected  to 
check  SiteScape  Forum  and  e- 
mail  every  two  days  and  post 

Because  of  the  nature  of  the 
evolving  discussion,  students 
should  be  constantly  engaged  in 
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Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

quality  contributions  at  least 
twice  a week.  Participation 
points  will  be  calculated  based 
on  these  postings. 

the  course,  without  any  lengthy 
absences  from  discussion. 
(Caldwell  & Taha,  1993) 

j 

SF 

j 

w 

A 

& 

The  instructor/ AI  should  make 
contact  with  students  who  are 
not  actively  participating  to  find 
out  why  and  address  their 
, concerns. 

Students  need  to  actively  feel  like 
they’re  part  of  the  community,  and 
that  the  instructor  is  interested  in 
their  well-being,  academic  or 
otherwise  (Palloff  & Pratt,  1999). 

i 

SF 

A 

w 

A 

Sr 

Students  will  work  together  at 
all  three  levels  of  interaction: 

•Discussion 

•Cooperation 

•Collaboration 

In  order  for  a newly-formed  cohort 
to  move  to  community,  they  must 
change  the  quality  of  their 
interactions.  The  community 
should  move  toward  successful 
use  of  collaboration,  in  addition  to 
the  continued  use  of  group 
discussions  and  cooperative  tasks. 

R511  Assignment  Specifications 

Based  on  the  existing  assignments  for  R51 1,  we  have  developed  a set  of  projects  and  assignments  that  will 
both  address  the  traditional  content  of  the  course,  and  build  community  based  on  the  discussion,  cooperation,  and 
collaboration  model.  Where  we  realize  that  collaboration  is  a more  rich  form  of  thinking  and  working  together,  we 
also  emphasize  the  necessity  for  students  to  work  at  all  three  levels  of  interaction  throughout  the  course. 

Discussion 

•At  the  beginning  of  the  semester,  students  will  be  divided  into  readings  discussion  groups  of  3-4  people.  For 
purposes  of  community  and  continuity,  they  will  remain  in  these  groups  throughout  the  semester. 

•In  SiteScape  Forum,  a team  will  be  created  for  each  readings  discussion  group.  The  group  will  manage  that  space, 
and  can  create  folders  for  each  week’s  readings  if  they  so  choose. 

•The  students  will  be  expected  to  discuss  the  week’s  readings  in  their  respective  folders.  Each  student  should  post  at 
least  twice  each  week. 

•The  role  of  facilitator  in  each  discussion  group  will  rotate  from  week  to  week.  The  facilitator  must  start  the 
conversation,  and  engage  group  members  to  participate. 

•The  role  of  summarizer  in  each  discussion  group  will  rotate  from  week  to  week.  At  the  end  of  the  week,  the 
summarizer  must  condense  the  group’s  main  discussion  points,  and  post  them  to  the  class  forum  (outside  the  team 
space). 

•There  will  be  a separate  folder  in  SiteScape  Forum  for  discussion  of  the  week’s  lecture  or  class  activity.  These 
posting  requirements  will  be  determined  at  a later  time,  dependent  on  the  format  of  the  course  lecture  material. 

Cooperation 
“Letter  Home”  Paper 

Students  will  review  each  other’s  papers  in  formative  stages.  For  the  “letter  home”  assignment,  students 
will  post  outlines  and  rough  drafts  by  set  deadlines,  and  a selected  group  of  peers  (ideally  from  outside  their  reading 
group)  will  have  to  read  and  give  feedback  on  them.  The  rationale  for  a number  of  small  deliverables  leading  to  the 
final  paper  is  that  distance  students  traditionally  need  regular  deadlines  and  prompt  feedback. 
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Final  Exam  Study  Guide 

Students  can  still  help  each  other  out  on  breaking  down  the  study  guide  and  elaborating  on  certain  sections 
of  it.  This  could  be  left  open  for  students  to  determine,  just  as  in  the  traditional  R511  class.  Simply  make  the 
suggestion  to  the  students  that  they  may  want  to  work  together  on  fleshing  out  the  study  guide,  and  leave  it  to  them 
to  decide  how  they  want  to  do  it. 

Collaboration 
ID  Model  Paper 

The  students  will  collaborate  on  the  ID  model  paper  as  in  the  traditional  R51 1 class  (using  their  reading  discussion 
teams  as  the  groups).  In  the  distance  version,  however,  it  will  be  critical  that  this  process  be  divided  into  small 
deliverables.  For  instance,  the  students  might  be  required  to  break  down  the  task  into  the  following  deadlines: 

• Week  One:  Each  group  member  must  post  initial  ideas  of  possible  models  to  evaluate  or  create.  This  is  not  in 
any  formal  structure  -just  a brain  dump.  Each  group  member  must  read  and  respond  to  the  discussion. 

• Week  Two:  Group  must  decide  on  a model  and  begin  explicating  the  model  and  describing  its 
strengths/weaknesses.  All  group  members  should  be  posting  during  this  week. 

• Week  Three:  Someone  in  the  group  should  summarize  the  discussion  into  a paper  outline.  Another  group 
member  should  develop  a paper  draft.  One  or  two  group  members  should  make  suggested  changes  and 
revisions.  The  final  group  member  should  write  the  final  draft  and  post  it. 

“ism”  Debate 

Students  will  participate  in  group  debates  revolving  around  behaviorism,  cognitivism,  and  constructivism. 
Ideally,  students  will  be  placed  into  3 groups  that  are  different  from  their  reading  groups.  Each  group  will  be 
assigned  one  of  the  “isms”  to  represent  in  the  debate.  Here,  a proposed  timeline  for  the  debate: 

• Week  One:  Individuals  will  write  an  informal  short  paper  (one  page  - perhaps  even  as  a bulleted  list) 
highlighting  the  major  strengths  of  their  “ism”  as  it  applies  to  distance  education  courses  and  will  post  it  for 
their  teammates.  Next,  the  team  will  enumerate  possible  rebuttals  from  the  other  groups  and  responses  to  those 
arguments.  The  first  week’s  discussion  and  postings  will  all  take  place  inside  a new  folder  established  for  that 
team. 

• Week  Two:  One  student  from  each  group  will  post  an  argument  about  why  their  position  is  the  best  to  a debate 
folder  open  to  the  whole  class.  Each  group  will  respond  to  each  of  the  other  groups. 

• Week  Three:  Debate  will  continue. 

• Week  Four : Each  individual  will  write  a brief  reflection  on  how  their  opinion  changed  throughout  the  debate. 


Checklist  for  R511  Instructor/Mentor 

Orientation 

□ Attend  Sunday  night  dinner  with  new  DE  students. 

Q Participate  in  2-hour  R51 1 class  welcome  session. 

Beginning  of  Semester 

□ Create  teams  in  SiteScape  Forum  for  each  readings  discussion  group.  Using  a naming  structure  like 
jewels  (Opal  team,  Ruby  team,  etc.)  is  an  easy  identifying  factor. 

o Divide  students  into  the  groups  evenly.  Make  sure  the  instructor  and  GA  are  listed  as  members  of 
all  teams. 

□ In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R51 1 Lecture  and  Class  Activity 
Discussion.” 

□ In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R51 1 Resources  & Tidbits” 

□ Create  a class  majordomo. 

Weekly 

Q Check  that  all  class  members  have  posted  at  least  twice  about  the  readings. 

o If  not,  make  decision  about  contacting  that  person  via  e-mail. 

Q Check  that  all  class  members  have  posted  about  the  lecture/class  activity, 
o If  not,  make  decision  about  contacting  that  person  via  e-mail. 


Q Post  some  comments  to  the  Online  Cafe.  This  could  be 
o News  stories 
o Responses  to  other  students 
o Encouragement 
o Personal  comments 
o IST/DE  news 

Q Reply  to  at  least  2 postings  a week,  to  encourage  students  to  post  thoughtful  responses  and  to  show  that 
you  are  present  and  actively  following  the  discussions. 

Before  the  “ism”  Project 

O In  SiteScape  Forum,  create  the  following  three  teams:  Behaviorism,  Cognitivism,  and  Constructivism, 
o Divide  students  equally  among  the  three  teams.  Make  sure  the  instructor  and  GA  are  listed  as 
members  of  all  teams. 

Before  the  “Letter  Home”  Paper 

O In  SiteScape  Forum,  create  the  following  five  teams:  Peer  Review  Group  1,  Peer  Review  Group  2,  Peer 
Review  Group  3,  Peer  Review  Group  4,  and  Peer  Review  Group  5. 

o Divide  students  equally  among  the  five  teams.  Make  sure  the  instructor  and  GA  are  listed  as 
members  of  all  teams. 

R521/522  Section 

Description  of  R521/522  (from  course  syllabus) 

R52 1/522,  Instructional  Design  and  Development,  is  a four-credit  course  that  has  historically  been  offered 
each  fall  semester  in  an  onsite  format.  This  course  is  required  by  all  1ST  MS  students  and  is  typically  taken 
concurrently  with  R51 1,  Instructional  Technology  Foundations  I,  and  R580, 1ST  Colloquium.  It  is  team-taught  by  at 
least  two  faculty  members  and  one  or  two  graduate  assistants  who  have  taken  the  course  themselves. 

Major  content  and  experience  objectives  of  R52 1/522: 

• Knowledge  of  instructional  design  principles 

• Knowledge  and  application  of  the  ADDIE  model  of  instructional  design  and  development 

• Understanding  and  application  of  simple  formative  evaluation  processes 

• Ability  to  recognize  and  employ  fundamental  principles  and  experiences  in  team-based  approach  to  project 
work 

Pedagogical  methods  used  in  R521/522: 

• Task-oriented  learning  through  “authentic”  projects 

• Diverse,  team-based  project  groups 

• Mentor/coach-based  instruction  for  project  team  support 

• Structured  timeline  of  deadlines  and  deliverables 

• Independent  learning,  i.e.,  students  take  responsibility  for  their  own  learning 

• Assignments  with  specific  criteria  that  engage  students  in  learning  specific  course  content,  with  leeway  given 
for  students  to  identify  their  own  topics 

Most  of  the  learning  in  the  course  occurs  within  the  context  of  projects  and  situations  similar  to  those  that 
instructional  designers  encounter  in  professional  work.  Projects  are  sequenced  such  that  the  processes  and  principles 
learned  in  the  first  ones  provide  foundation  of  understanding  and  competence  for  progressively  more  complex  ones 
that  follow.  This  progression  of  increasingly  elaborated  projects  continues  through  the  academic  year  into  R561, 
Evaluation  and  Change  Management,  and  is  intended  to  carry  on  throughout  the  student’s  academic  experiences  in 
completing  the  1ST  MS  program. 

In  the  onsite  version  of  R52 1/522,  class  meetings  occur  twice  per  week  in  2.5  -hour  sessions.  Class  sessions 
involve  one  or  more  of  a variety  of  activities,  including  lectures  or  presentations  about  specific  topics,  readings 
discussions,  project  group  meeting  time,  group  project  presentation,  or  hands-on  design  activities.  Directed  readings 
compiled  in  a course  packet  are  referenced  as  practical  resources  to  support  projects  and  class  discussion  activities  in 
the  course. 

The  instructors  believe  that  people  learn  best  when  they  are  highly  motivated  and  actively  engaged  in 
learning  tasks,  that  learning  is  most  useful  when  it  is  directly  related  to  learner  needs.  Thus,  students  are  expected  to 
take  responsibility  for  their  own  learning.  The  course  begins  with  a fair  amount  of  guidance  from  the  instructors,  in 
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terms  of  what  information  to  access  and  how  to  facilitate  personal  learning,  then  gradually  decreases  that  guidance 
to  require  students  to  actively  seek  resources  on  their  own  to  perform  the  assigned  tasks. 

Major  projects  in  R52 1/522  are  completed  by  groups  of  three  students,  each  mentored  by  an  assigned 
instructor  “coach.”  To  perform  most  satisfactorily  in  the  course,  students  must  spend  many  hours  per  week  outside 
of  class  developing  and  completing  these  projects.  At  the  completion  of  a project,  each  member  of  a given  group  is 
awarded  the  same  grade  (a  “group  grade”)  as  his/her  teammates.  Approximately  twenty  percent  of  that  grade  is 
awarded  for  the  deliverable  produced  in  the  project  (e.g.,  the  instructional  tool  developed  and  a design  report), 
whereas  the  remaining  portion  of  the  grade  is  awarded  according  to  the  way  members  worked  within  the  team 
setting.  Some  students  come  into  the  program  with  extensive  background  in  true  teamwork,  but  most  do  not.  Thus, 
the  instructors  devote  a portion  of  instructional  time  early  in  the  semester  toward  preparing  students  for  the  team 
experience.  Throughout  the  duration  of  each  project,  group  coaches  continue  to  offer  advice  and  guidance  for  the 
team  process. 


R52 1/522  Instructional  Strategies  and  Rationales 


Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

A 

W 

Learners  will  be  divided  into  3- 
4 groups  for  discussion  of 
readings  and  course  projects. 

Small  groups  facilitate  better 
discussion.  (Hiltz,  1998) 

i 

SF 

A 

W 

A 

m 

A 

W 

Provide  criteria  that  define 
appropriate  course  topics, 
leaving  room  for  choice  and 
opportunities  to  leverage  work- 
related  projects  as  course 
projects. 

• Learners  select  a topic  and 
procedure  for  project. 

• Each  team  selects  4 
readings  to  read  and 
summarize  for  the  whole 
class  (for  discussion) 

In  order  to  build  community, 
learners  need  ownership.  (Schwier, 
in  press) 

A 

w 

A 

SF 

i 

W 

Each  week,  someone  from  each 
group  will  summarize  their 
group’s  discussion  and  post  the 
results  for  the  other  groups  to 
read. 

Bringing  from  small  groups  to  the 
larger  group  provides  for  more 
viewpoints  and  better  discussion. 

A 

W 

A 

SF 

A 

SF 

Students  will  be  divided  into  3- 
4 small  groups  for  discussion  of 
readings  and  course  projects. 

Small  groups  facilitate  better 
discussion  (Hiltz,  1998)  for 
learner-material  interaction. 

A 

& 

A 

W 

A 

fi? 

Instructional  activities  will  require  the 
learner  to  actively  participate  in  the 
acquisition  and  processing  of 
educational  content. 

• Team-based  authentic  projects 
where  the  learners  learn  by  doing. 

• Discussing  readings  online  and 

To  better  assimilate  and  process 
what  they  have  learned,  students 
require  a forum  to  critically  reflect 
on  the  material  and  on  themselves 
as  learners  (Palloff  & Pratt,  1999). 
Keeping  in  touch  with  the 
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Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

role-playing. 

• Require  high-quality  online 
interactions  with  peers  and 
discussions  of  readings  by 
making  a portions  of  the  grade 
dependent  on  it  (25% 
recommended) 

• Groups  will  be  responsible  for 
posting  some  of  their  work  in 
progress  (e.g.,  each  group  posts  a 
description  of  a different  aspect  of 
ADDIE). 

• Instructor  should  use  open-ended 
questions  to  encourage  dialogue. 

• Ask  learners  to  provide  URLs  that 
enhance  learning. 

professor  improves  learner-faculty 
interaction. 

j 

SF 

4 

SP 

Instructor  will  phone  each 
learner  before  class  begins. 
(Spear  & Bruce,  1997) 

One-on-one  verbal  communication 
between  learner  and  instructor 
solidifies  relationship. 

i 

SP 

i 

SP 

Establish  a virtual  office  hour: 
one  hour  where  instructor  will 
be  available  for  online  chats, 
office  phone  calls,  or  e-mail. 
Inform  learners  of  the  faculty 
member’s  expected  e-mail  or 
voicemail  response  time,  e.g., 
within  24  hours,  twice  a week, 
etc.  (Spear  & Bruce,  1997) 
(Spear  & Bruce,  1997) 

Students  like  to  know  the 
professor  is  available  at  a 
particular  time  to  address  e-mail 
concerns. 

j 

i 

fip 

Instructor  will  be  proactive, 
following  up  on  the  learner  who 
is  not  participating  in  chats, 
discussions,  etc. 

In  the  distance  format,  it  is  easy 
for  students  to  lose  touch  with  the 
class  and  slowly  drop  out.  Active 
intervention  from  the  instructor 
can  lessen  attriction. 

4 

fip 

i 

i 

w 

4 

fip 

Conduct  a phone  conference 
with  each  team  at  least  once 
during  the  development  of  each 
project. 

Verbal  communication  between 
the  team  and  instructor  solidifies 
relationship  and  makes  for  easier 
clarification. 

i 

s* 

i 

i 

fip 

Invite  other  professors  to  lead 
discussions  (Davies  and 
Reigeluth  on  concept  learning) 
where  learners  can  interact 

Students  will  appreciate  input 
from  experts  in  the  field.  The  will 
feel  less  isolated  from  the  rest  of 
the  department  when  they  can 

Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Parti 

Strategies 

Rationale 

directly  with  experts  to  deepen 
understanding. 

interact  with  other  instructors. 

R521  Assignment  Specifications 

Based  on  the  existing  assignments  for  R521,  we  have  developed  a set  of  projects  and  assignments  that  will 
both  address  the  traditional  content  of  the  course  and  build  community  based  on  the  discussion,  cooperation,  and 
collaboration  model  The  students  will  collaborate  on  production  projects,  discuss  readings  or  lecture  topics,  and 
reflect  on  activities  and  experiences  throughout  the  course.  While  we  realize  that  collaboration  is  the  richest  form  of 
thinking  and  working  together,  we  also  emphasize  the  necessity  for  students  to  work  at  all  three  levels  of  interaction 
throughout  the  course. 

Discussion 

Readings 

Discussion  activities  centered  around  the  course’s  major  themes  (e.g.,  usability,  design,  evaluation). 

• At  the  beginning  of  the  semester,  students  will  be  divided  into  readings  discussion  groups  of  5 people 
(different  from  those  in  their  project  groups).  For  purposes  of  community  and  continuity,  they  will  remain  in 
these  groups  throughout  the  semester. 

• In  SiteScape  Forum,  a team  will  be  created  for  each  readings  discussion  group.  The  group  will  manage  that 
space,  and  can  create  folders  for  each  week’s  readings  if  they  so  choose. 

• The  students  will  be  expected  to  discuss  the  assigned  readings  in  their  respective  folders.  Each  student  should 
post  at  least  twice  each  week. 

• The  role  of  facilitator  in  each  discussion  group  will  rotate  from  assignment  to  assignment.  The  facilitator  must 
start  the  conversation,  and  engage  group  members  to  participate. 

• The  role  of  summarizer  in  each  discussion  group  will  rotate  from  assignment  to  assignment.  At  the  end,  the 
summarizer  must  condense  the  group’s  main  discussion  points,  and  post  them  to  the  class  forum  (outside  the 
team  space). 

Cooperation 

Group  projects 

• At  the  beginning  of  each  project,  students  will  be  divided  into  groups  of  three.  Each  group  will  work 
collectively  to  complete  its  own  project.  A team  “coach”  (an  instructor  or  graduate  assistant)  will  be  assigned  to 
each  group  to  offer  advice  and  guidance  for  the  team  process. 

• In  SiteScape  Forum,  a team  will  be  created  for  each  project  group  (including  the  course  instructors  and 
mentors).  The  group  will  manage  that  space. 

• Groups  will  be  required  to  post  all  team  meeting  summaries  and  other  artifacts  of  their  team  processes  on  the 
forum. 

Collaboration 

Group  projectsFor  each  project,  the  team  will  be  intentionally  diverse  in  gender,  nationality  and/or  job 
background  as  much  as  possible  to  encourage  multiple  points  of  view. 

• Projects  will  be  assigned  group  grades,  a large  portion  of  which  is  assigned  to  the  “group  process.” 

• Give  project  rubrics,  teams  will  be  encouraged  to  brainstorm  possible  topics  and  come  to  consensus  to  identify 
their  own  topics  for  projects. 

• Teams  will  engage  in  formative  peer  reviews  of  each  others’  projects  and  materials  for  projects  throughout  the 
course. 

• Lectures  and  course  topics  will  be  presented  by  different  instructors  throughout  the  course,  providing  a model 
of  collaboration  for  students. 
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Checklist  for  R521  Instructor/Mentor 

Orientation 

□ Attend  Sunday  night  dinner  with  new  DE  students. 

Q Coordinate  a team-based  project  that  emulates  the  required  peer  interaction  and  interdependence  and 
time-limited  working  tensions  of  R521  production  projects. 

Q Post  expectations  (time,  participation,  assignments,  dates) 

Beginning  of  Semester 

Q Create  teams  in  SiteScape  Forum  for  each  readings  discussion  group. 

o Divide  students  into  the  groups  evenly.  Make  sure  the  instructor  and  GA  are  included  as  members 
of  all  teams. 

Q In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R521  Lecture  and  Class  Activity 
Discussion.” 

□ In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R521  Resources  & Tidbits” 

□ Create  a class  majordomo  listserv  and  direct  all  class  members  to  subscribe  to  it. 

Weekly 

Q Post  some  comments  to  the  Online  Cafe.  This  could  be 
o News  stories 
o Responses  to  other  students 
o Encouragement 
o Personal  comments 
o This  week  in  1ST 
o Post  reflection  questions  each  week 
Beginning  of  Each  Project 

Q Create  teams  in  SiteScape  Forum  for  each  project  group. 

o Divide  students  into  the  groups  evenly.  Make  sure  the 
of  all  teams. 

Q Direct  each  team  to  construct  and  post  its  own  individualized 
team  process  and  completing  its  project. 

Throughout  Project 

Q Check  that  each  project  team  is  posting  evidence  of  cooperative  work  on  project  at  least  once  per  week. 

o If  not,  make  decision  about  contacting  that  group  via  e-mail. 

Q Reply  to  at  least  1 or  2 postings  a week  per  group,  to  encourage  students  to  post  thoughtful  responses  and 
to  show  that  you  are  present  and  actively  following  the  discussions 
Q Check  that  all  team  members  are  participating  at  least  once  every  two  weeks  within  their  own  project 
teams 

End  of  Project 

Q Review  reflection  essays  from  each  team  member  about  lessons  learned  from  the  production  and  team 
processes 

Q Collect  peer  grading  of  team  members’  participation  within  each  team 

Evaluation 

The  final  step  in  the  creation  of  a learning  community  in  these  courses  is  to  evaluate  whether  such  a 
community  has  formed  and,  if  so,  in  what  ways  the  community  aspect  has  contributed  to  learning.  We  are  basing  our 
strategies  for  evaluating  the  success  of  community-building  in  these  courses  on  Palloff  & Pratt’s  (1999)  indicators 
that  an  online  community  has  been  forming: 

• Active  interaction  involving  both  course  content  and  personal  communication. 

• Collaborative  learning  evidenced  by  comments  directed  primarily  student  to  student  rather  than  student  to 
instructor. 

• Socially  constructed  meaning  evidenced  by  agreement  or  questioning,  with  the  intent  to  achieve  agreement 
among  students. 

• Sharing  of  resources  among  students 


instructor  and  GA  are  included  as  members 
strategies  and  timeline  for  conducting  the 
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• Expressions  of  support  and  encouragement  exchanged  between  students,  as  well  as  willingness  to  critically 
evaluate  the  work  of  others,  (p.  32) 

The  course  evaluations  will  take  two  forms:  Formative  evaluations  are  undertaken  throughout  the  course  so 
that  necessary  adjustments  in  course  delivery  and  activities  can  be  identified  and  made.  Summative  evaluations  are 
performed  at  the  conclusion  of  the  course  to  measure  final  learning  outcomes  and  student  satisfaction.  Both  forms 
provide  fundamental  indicators  of  the  overall  success  of  the  course  and  its  participants  in  meeting  the  initially  stated 
objectives.  Pallofif  & Pratt  recommend  employing  evaluations  over  three  distinct  elements  of  a online  course: 
student  performance  and  learning,  effectiveness  of  the  course  in  supporting  student  learning  objectives,  and  overall 
student  experiences  of  the  students  in  working  in  an  online  environment.  For  our  purposes  of  assessing  community 
formation,  the  emphasis  on  student  performance  is  most  the  most  important  factor  on  which  to  focus. 

We  have  stated  already  that  two  key  indicators  that  the  evolution  of  a community  has  occurred  are  evidence 
of  participants  accepting  ownership  of  the  community  and  realizing  a shared  identity.  The  metaphor  of  scaffolding 
activities  and  course  strategies  as  mechanisms  to  foster  community  implies  that  the  instructor  provides  models  and 
activities  to  course  members  through  which  they  exercise  community-like  tasks  and  interactions.  These  scaffolds  are 
erected  as  temporary  measures  to  support  the  desired  behavioral  outcomes  until  observed  behavior  indicates  they  are 
no  longer  used  or  needed,  then  they  are  gradually  removed.  Concurrently,  formative  evaluation  that  measures 
indicators  of  the  extent  to  which  online  community  is  occurring  becomes  the  key  factor  in  determining  the  necessity 
and  lifespan  of  each  scaffolding  device. 

Suggested  methods  for  formatively  assessing  the  level  of  online  community  throughout  the  duration  of  the 
courses  are  as  follows: 

• Continually  monitor  the  amount,  type  and  effectiveness  of  discussion  in  all  media,  particularly  student-to- 
student  discussion 

• Administer  periodic  interviews  and  web-based  questionnairesto  students  to  gather  qualitative  feedback  about 
reactions  to  the  level  of  community  they  are  experiencing  and  its  usefulness  to  their  learning 

• Look  for  evidence  within  all  communication  media  of  resource  sharing  and/or  inter-community  encouragement 
or  support 

• Compare  progression  of  reflective  essays  of  students  to  identify  evolution  of  self-assessments  that  indicate 
personal  commitment  to  the  community  or  deepening  of  learning  and  thought  about  key  issues  discussed 
among  members 

We  do  not  anticipate  that  a mature  community  will  have  been  generated  from  this  one  semester  alone. 
However,  we  do  expect  that  the  R521  and  R51 1 experiences  of  these  students  will  create  a solid  foundation  of  an 
infant  community  that  will  continue  evolving  throughout  their  career  in  the  1ST  DE  MS  program.  Summative 
evaluation  in  the  context  of  assessing  community  building  is  useful  for  determining  the  overall  effectiveness  of  the 
online  community  environment  on  the  students’  experiences  both  during  these  courses  and  in  future  ones. 

Suggested  methods  of  summative  evaluation  are  as  follows: 

• Compare  pre-and  post-course  attitudes  of  students  regarding  confidence  with  working  collaboratively  with  a 
distributed  or  online  project  team 

• Compare  pre-  and  post-course  opinions  of  students  regarding  their  comfort  levels  with  and  reactions  to 
collaborative  projects 

• Assign  a final  reflective  essay  in  which  students  describe  a personal  action  plan  for  applying  the  experiences 
and  knowledge  gained  through  the  course,  specifically  those  relating  to  collaboration  and  communities 

• Perform  longer-range  (e.g.,  2-3  months  later)  follow  up  interviews  and  surveys  with  students  that  engage  them 
in  reflection  on  the  impact  of  community  and  collaboration  on  courses  taken  after  R52 1 and  r5 1 1 

Finally,  we  intend  these  strategies  of  evaluating  community  building  in  R521  and  R51 1,  although  holistic 
in  spirit,  merely  as  a framework  on  which  more  specific  and  precise  assessments  can  be  constructed.  We  believe 
deeper  exploration  of  success  factors  in  fostering  online  community  would  be  a very  fertile  topic  for  further  research 
and  warrants  further  investigation. 

Questions  for  Further  Research 

Beyond  the  evaluation  of  the  success  or  failure  of  community  in  the  Fall  2000  Distance  Masters  Core,  there 
are  other  topics  worthy  of  research. 

• What  are  some  valid  measures  of  community  development? 

• If  community  formed,  what  was  its  effect  on  the  learning? 

• How  can  learners  be  motivated  to  take  part  in  virtual  academic  or  social  community  activities? 
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• What  are  special  features  of  “forced  community”  like  the  Masters  cohort? 

• What  is  the  expected/observed  life  cycle  of  the  Distance  Masters  learning  community? 

• How  does  this  community  develop  and  maintain  its  history? 

• Should  the  Distance  community  be  integrated  with  the  residential  graduate  community?  If  so,  in  both  academic 
and  social  ways?  If  so,  how  can  this  be  accomplished? 

• How  can  the  community  best  be  mentored? 

• What  are  the  different  roles  for  instructors,  graduate  assistants,  volunteers,  upper-year  1ST  students,  etc? 

• What  communication/collaboration  tools  foster  the  development  of  a learning  community? 

• What  are  the  best  practices  for  using  existing  communication  tools  in  distance  education? 

• What  tool  features  lend  themselves  to  different  aspects  of  collaboration  and  community-building? 

• How  appropriate  were  the  tools  chosen  for  Fall  2000  in  terms  of  collaboration  and  community  formation? 

Conclusion 

Having  determined  that  richer  learning  takes  place  within  the  context  of  a learning  community,  this  report 
provides  background  descriptions  of  characteristics  of  community  and,  more  specifically,  a virtual  learning 
community.  We  discuss  the  goal  of  moving  a cohort  to  a learning  community  through  scaffolding  activities  rooted  in 
the  communication  formats  of  discussion,  cooperation,  and  collaboration. 

The  report  then  treats  the  Core  classes  in  three  separate  sections:  Core  (principally  orientation  and  the 
online  cafe),  R51 1,  and  R52 1/522.  The  courses  are  described,  instructional  strategies  and  rationales  are  presented, 
possible  assignments  are  detailed,  and  an  instructor  checklist  is  provided. 

Finally  we  thought  it  necessary  to  determine  some  strategies  to  evaluate  a)  whether  community  has  formed  within 
the  cohort,  and  b)  in  what  ways  the  community  contributed  to  deeper  learning.  We  also  provide  some  possible  topics 
for  further  study. 
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Lesson  by  Pre-service  Teachers 


G.  Brandon  Murry 
Francie  R.  Murry 
University  of  Northern  Colorado 


Abstract 

This  study  compared  the  use  of  two  developmental  alternatives:  a Web  Editor  (WE)  in  combination  with  a 
customized  template/ shell  (Teaching  Not  Teching,  T-N-T)  and  a Web  Editor  (WE)  only,  for  development  of  a Web- 
based  lesson  by  pre-service  teachers . Six  hypotheses  were  tested  to  find  whether  the  WE  + T-N-T  alternative  was 
more  efficient,  effective,  and  appealing  than  the  WE  only  development  alternative.  The  efficiency,  effectiveness,  and 
appeal  of  developing  a Web-based  lesson  was  examined  using  these  variables ; lesson  creation  time,  perceived 
effort,  inclusion  of  six  specified  instructional  components,  functionality  of  six  specified  technical  components, 
teacher  appeal,  and  likelihood  of  intended  future  use  (dependent  variables)  and  the  lessons  development  alternatives 
WE  + T-N-T  or  WE  only  (independent  variable). 

A quasi-experimental  design  and  t-test  analyses  were  employed.  Pre-service  teachers  (N  = 103)  were 
assigned  to  one  of  the  two  development  alternatives.  Instruments  included  researcher-developed  evaluations,  self- 
report  tools  and  the  Computer  Attitude  Scale  (CAS).  Dependent  variables  investigated  in  this  study  include:  lesson 
creation  time,  perceived  effort,  inclusion  of  six  specified  instructional  components,  functionality  of  six  technical 
components,  teacher  appeal,  and  rate  of  future  use.  Statistical  analysis  of  six  hypotheses  confirmed  that  developing 
Web-based  lessons  using  WE  + T-N-T  was  more  effective  than  using  WE  only.  The  analysis  also  revealed  that 
neither  developmental  alternative  was  superior  in  terms  of  efficiency  or  appeal. 

Purpose  of  the  Study 

A report  from  the  National  Center  for  Educational  Statistics  (NCES)  (1999)  details  that  although 
educational  technology  is  considered  a means  for  transforming  education,  only  20%  of  teachers  feel  well  prepared  to 
integrate  technology  into  classroom  instruction.  This  study  examines  a tool  for  promoting  technology  integration 
through  pre-service  teacher  skill  acquisition  and  application  of  Web-based  lesson  development  when  presented 
through  two  different  alternatives  (WE  + T-N-T  and  WE  only).  This  study  investigated  whether  using  a Web  editor 
(WE)  in  conjunction  with  a custom  created  template/shell  (T-N-T:  Teaching  Not  Teching)  is  more  efficient, 
effective,  and  appealing  for  pre-service  teachers  to  use  in  the  development  of  Web-based  lessons  than  using  a Web 
editor  only.  The  WE  used  in  this  study  was  Microsoft  FrontPage  2000. 

Each  pre-service  teacher  created  a Web-based  lesson  using  one  of  two  lesson  development  alternatives.  The 
Web-based  lessons  were  then  evaluated  using  three  criteria:  efficiency,  effectiveness,  and  appeal.  Comparison  of  16 
variables/sub-variables  was  made  between  lessons  created  with  WE  + T-N-T  and  WE  only. 

The  three  criteria  for  usability  was  applied  in  the  evaluation  phase  to  determine  if  one  Web-based  lesson 
development  alternative  provides  better  support  for  pre-service  teachers  compared  to  another  alternative.  Teachers 
must  have  access  to  technological  innovations  that  help  them  meet  the  needs  of  a diverse  audience,  get  the  job  done 
quickly,  and  have  the  ability  to  integrate  best  practices  into  the  interface  design  and  lesson  content. 

Description  of  the  Sample 

During  the  1999-2000  Fall  Semester,  161  pre-service  teachers  were  enrolled  in  four  sections  of  ET  347 
Educational  Technology  Applications  for  Elementary  Teaching,  one  section  of  ET  348  Educational  Technology 
Applications  for  Middle  Grades  Teaching,  and  three  sections  of  ET  349  Educational  Technology  Applications  for 
Secondary  Teaching.  All  of  these  students  were  invited  to  participate  in  this  study.  Permission  to  participate  was 
received  from  a total  of  144  students.  Each  student  completed  the  Demographics  Questionnaire  and  the  Consent 
Form  during  the  first  class  session.  Of  the  144  students  in  the  study,  half  were  identified  as  the  Control  group  while 
the  remaining  students  were  identified  as  the  Treatment  group. 

The  groups  were  selected  by  nonrandom  methods;  however,  the  intact  sections  were  randomly  assigned  to 
treatment  groups.  Two  of  the  four  sections  of  ET  347  (elementary  pre-service  teachers)  were  randomly  assigned  to 
the  Control  group  and  the  remaining  two  sections  were  assigned  to  the  Treatment  group.  Two  groups  of  ET  349 
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(secondary  pre-service  teachers)  were  drawn  and  assigned  to  the  Control  and  one  remaining  section  of  ET  349  and 
one  section  of  ET  348  (middle  grades  pre-service  teachers)  were  assigned  to  the  Treatment  group. 

Forty-one  of  the  144  students  (28.47%)  who  had  agreed  to  participate  in  the  study  either  did  not  complete 
their  Web-based  lesson  or  other  required  instruments  and  subsequently  were  eliminated  from  the  study.  Thus,  this 
study’s  sample  consisted  of  103  students,  that  is,  64%  of  the  Educational  Technology  students  who  initially  had 
been  invited  to  participate.  Final  group  size  for  the  Control  group  was  52  and  for  the  Treatment  group  was  51 . 

The  intact  class  sizes  ranged  from  16  to  26  students.  The  classes  were  scheduled  for  50  minutes  and  met  on 
the  following  days  and  times;  two  Monday  classes  at  12:20  and  1:25  PM,  one  Tuesday  class  at  9:05  AM,  three 
Wednesday  classes  at  8:00  AM,  12:20,  and  1:25  PM,  one  Thursday  class  at  6:00  PM,  and  one  Friday  class  at  10:10 
AM. 

Research  Questions 

The  research  questions  were  centralized  around  the  usability/evaluation  criteria  efficiency,  effectiveness, 
and  appeal  as  follows: 

Efficiency 

RQi:  Are  pre-service  teachers  able  to  develop  a Web-based  lesson  in  less  time  when  they  use  WE  + T-N-T  than 

when  they  develop  a Web-based  lesson  using  WE  only? 

RQ2:  Will  pre-service  teachers  perceive  that  less  effort  is  required  by  using  WE  + T-N-T  than  when  they  develop 
Web-based  lessons  using  WE  only? 

Effectiveness 

RQ3:  Are  the  six  specified  instructional  components  (lesson  goal(s),  objectives  written  in  performance-based  terms, 
student  performance,  student  performance  evaluation,  student-to-teacher  contact,  and  location  cues  in  the  site) 
present  when  pre-service  teachers  develop  a Web-based  lesson  using  WE  + T-N-T  and  when  using  WE  only? 

RQ4:  If  present,  which  of  the  six  specified  technical  components  (navigation,  image  presence,  mailto  links, 
interactive  mechanism,  audio  and  video)  are  functional  when  pre-service  teachers  develop  a Web-based  lesson  using 
WE  + T-N-T  and  when  using  WE  only? 

Appeal 

RQ5:  What  degree  of  teacher  appeal  is  elicited  from  pre-service  teachers’  interaction  with  the  development  process 
of  a Web-based  lesson  using  WE  + T-N-T  and  from  pre-service  teachers  who  use  WE  only? 

RQ6:  Do  pre-service  teachers  intend  to  continue  Web-based  lesson  development  with  the  lesson  development 
alternative  they  used  in  the  study  in  their  future  instructional  settings? 

Research  Design 

This  study  is  quasi-experimental  (Smith  & Glass,  1987).  That  is,  the  independent  variable  is  an  introduced 
treatment,  although  there  will  not  be  total  control  over  which  participants  receive  which  treatment.  Participants  were 
selected  by  nonrandom  methods  and  then  the  intact  sections  were  randomly  assigned  to  treatment  groups.  This 
research  paradigm  may  be  represented  as  follows: 

Group  I R Oi  02 

WE  only 


Group  II  R Oi  X O2 

WE  + T-N-T 

R = Random  assignment  of  treatment  to  intact  classes,  Consent  Form,  Demographic  Questionnaire 
Oi  = Instruction  on  how  to  use  the  Web  editor 
X = Intervention  treatment  (T-N-T) 

O2  = Work  Time-Log,  Effort  Questionnaire,  Instructional  Components  Evaluation  Checklist,  Technical 
Components  Evaluation  Checklist,  Computer  Attitude  Scale  (CAS),  and  Intended  Future  Use 
Questionnaire,  Summary  Data  Sheet 
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Data  Analysis 

Results  regarding  the  evaluation  of  the  Web-based  lessons  created  with  a Web  editor  combined  with  a 
template/shell  structure  from  "Teaching-Not-Teching"  (T-N-T)  and  those  created  using  a Web  editor  (WE)  only 
developed  by  pre-service  teachers  are  presented  in  this  section.  The  evaluation  was  based  on  the  comparison 
criterion  of  efficiency,  effectiveness  and  appeal.  These  variables  were  comprised  of  16  sub-variables. 

The  t-test  was  used  to  analyze  the  data  and  determine  if  there  was  a significant  difference  between  the 
Control  and  Treatment  groups  as  measured  by  the  16  identified  sub-variables.  This  study  used  six  independent  t- 
tests;  therefore  the  probability  of  one  or  more  Type-I  errors  was  greater  than  the  alpha  set  for  any  single  t-test.  Due 
to  the  multiple  t-test  error  rate  occurring  from  six  t-tests,  an  alpha  level  of  .0167  was  set  for  each  test  resulting  in  an 
overall  experiment-wise  alpha  of  not  more  than  .10.  Given  the  exploratory  nature  of  the  study,  this  error  range  was 
judged  appropriate.  The  research  null  hypotheses  are  presented  with  the  related  analyses  and  results. 

H0i:  There  is  no  statistically  significant  difference  between  length  of  time  required  to  develop  a Web-based  lesson 
using  WE  + T-N-T  compared  to  using  WE  only. 

Based  on  the  results  of  the  independent  t-test,  H0i  was  retained,  indicating  there  was  no  difference  in  the 
time  required  by  pre-service  teachers  to  create  a Web-based  lesson  using  WE  + T-N-T  than  when  using  WE  only  (t 
= -0.9457,  p = 0.1733).  Data  were  compiled  and  analyzed  from  the  Work  Time-Log.  The  results  are  presented  in 
Table  1. 


Table  1.  Work  Time-Loz 


Group 

Mean  Time  (minutes) 

SD  (minutes) 

df 

P 

Treatment 

519.1764 

215.3836 

101 

0.1733 

(n  = 51) 

Control 

476.5384 

241.1787 

cT 

cn 

ll 

3 

Participants  in  the  study  used  the  Work  Time-Log  to  record  the  number  of  minutes  they  spent  designing 
and  developing  their  Web-based  lesson.  The  six  specific  tasks  itemized  on  the  Work  Time-Log  were:  1)  Tinkering 
with  the  computer  program,  2)  Storyboarding,  3)  Collecting  images,  audio,  video,  and  other  cool  stuff,  4)  Thinking 
about  my  Web-based  lesson,  5)  Collecting  or  creating  lesson  content,  and  6)  Developing  Web-based  lesson  with  the 
editor.  Although  there  was  not  statistical  significance  between  the  total  number  of  minutes  participants  from  each 
group  took  to  complete  their  lesson,  participants  in  the  Treatment  group  spent,  on  average,  43  more  minutes  than  did 
the  participants  from  the  Control  group. 

H02:  There  is  no  statistically  significant  difference  between  perceived  required  effort  when  using  WE  + T-N-T 
compared  to  when  WE  only  is  used  by  pre-service  teachers  to  develop  a Web-based  lesson. 

H02  was  retained,  suggesting  no  difference  in  the  perceived  effort  by  pre-service  teachers  when  developing 
a Web-based  lesson  using  WE  + T-N-T  than  when  using  WE  only  (t  = 1 .8673,  p = 0.0324).  Data  for  this  comparison 
were  compiled  from  the  Effort  Questionnaire.  The  results  of  this  analysis  are  found  in  Table  2 


Table2.  Perceived  Effort 


Group 

Mean  Effort 

SD 

df 

P 

Treatment 
(n  = 51) 

7.0588 

1.9226 

101 

0.0324 

Control 
(n  = 52) 

6.3750 

1.7928 

H03:  There  will  be  no  statistically  significant  difference  between  the  frequency  of  presence  of  the  six  specified 
instructional  components  (e.g.,  lesson  goal(s),  objectives  written  in  performance-based  terms,  student  performance 
activity,  student  performance  evaluation,  student-to-teacher  contact,  and  location  cues  in  the  site)  in  a Web-based 
lesson  when  developed  by  pre-service  teachers  using  WE  + T-N-T  than  when  developed  using  WE  only. 
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H03  was  rejected,  indicating  a significant  statistical  difference  between  the  frequency  of  presence  of  the  six 
specified  instructional  components  in  a Web-based  lesson  developed  by  pre-service  teachers  utilizing  WE  + T-N-T 
than  when  developed  using  WE  only  (t  = 18.5048,  p = 0.0000).  Data  were  derived  from  a scale  that  ranged  from  a 
low  of  zero  points  to  a high  of  six  points.  The  data  indicated  a statistical  favor  for  the  WE  + T-N-T  group.  See  Table 
3 to  review  the  results. 


Table3 . Instructional  Components 


Group 

Mean  Score 

SD 

df 

P 

Treatment 

5.3333 

0.9933 

101 

0.0000* 

(n  = 5 1 ) 

Control 

2.1153 

0.7586 

(n  = 52) 

* = a < 0.0167 


The  six  specific  instructional  components  itemized  on  the  Instructional  Components  Evaluation  Checklist 
are  as  follows:  1)  Lesson  goal(s),  2)  Objectives  written  in  performance-based  terms,  3)  Student  performance 
activity,  4)  Student  performance  evaluation,  5)  Student-to-teacher  contact,  and  6)  User-location  within  the  site  cues. 
The  Treatment  group  scored  higher  than  the  Control  group  on  all  six  instructional  components.  Not  one  Web-based 
lesson  developed  by  the  participants  in  the  Control  group  included  Objectives  written  in  performance-based  terms, 
or  Student  performance  activity,  or  Student  performance  evaluation. 

Some  participants  from  the  Control  group  and  the  Treatment  group  designed  what  resembled  an  activity 
and/or  quiz  in  their  Web-based  lesson  but  did  not  receive  points  for  these  components  on  the  Instructional 
Components  Evaluation  Checklist.  Operational  definitions  of  these  three  variables  follow. 

Objectives  written  in  performance-based  terms:  refers  to  a statement  that  describes  what  learners  should  be  able  to 
do  when  they  have  completed  the  lesson.  "What  learners  'do'  must  be  observable  so  that  the  learners  know  that  they 
have  learned  and  what  they  have  learned"  (Smith  & Ragan,  1993,  p.  91). 

Student  performance  activity:  refers  to  eliciting  specific  behavior  from  the  student  based  on  the  performance 
objectives  for  the  lesson. 

Student  performance  evaluation:  refers  to  supplying  the  student  with  feedback  on  their  performance  based  upon 
student  performance  activity. 

The  protocol  in  awarding  points  for  these  three  variables  was  to  first  look  for  objectives  written  in 
performance-based  terms.  The  performance  objectives  are  the  foundation  for  the  other  two  variables  and  if  there 
were  no  performance  objectives  contained  in  the  lesson  there  could  be  no  points  awarded  for  either  "student 
performance  activity"  or  "student  performance  evaluation."  As  stated  a-priori  in  the  definition  of  “student 
performance  activity,”  performance  was  based  upon  stated  objectives  and  if  objectives  did  not  exist  in  the  lesson  no 
points  were  awarded  for  student  performance  activity.  Furthermore,  no  points  could  be  awarded  for  student 
performance  evaluation,  because  as  stated  a-priori  in  the  definition,  “student  performance  evaluation  is  to  be  based 
upon  student  performance  activity.” 

H04:  There  will  be  no  statistically  significant  difference  between  the  frequency  of  functionality  when  the 
six  specified  technical  components  (e.g.,  navigation,  image  presence,  mailto  links,  interactive  mechanism,  audio  and 
video)  are  developed  by  pre-service  teachers  utilizing  WE  + T-N-T  than  when  developed  using  WE  only. 

H04  was  rejected,  indicating  significant  statistical  difference  between  the  frequency  of  functionality  of  the 
six  specified  technical  components:  1)  Navigation,  2)  Image  presence,  3)  Mailto  links,  4)  Interactive  mechanism,  5) 
Audio,  and  6)  Video  in  Web-based  lessons  when  developed  by  pre-service  teachers  utilizing  WE  + T-N-T  than 
when  developed  using  WE  only  (t  = 9.7680,  p = 0.0000).  Data  are  derived  from  a scale  that  ranged  from  a low  of 
zero  points  to  a high  of  six  points.  The  statistical  preference  was  in  favor  of  the  WE  + T-N-T. 

T-N-T  group.  The  results  can  be  found  in  the  following. 
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Table  4 . Technical  Components 


Group 

Mean  Score 

SD 

df 

P 

Treatment 

3.4901 

0.6744 

101 

0.0000* 

II 

a 

Control 
(n  = 52) 

2.0769 

0.7883 

* = a < 0.0167 


The  scores  on  the  Technical  Components  Evaluation  Checklist  were  based  on  functionality  of  the  six 
specified  technical  components.  Not  one  participant  from  either  the  Treatment  or  the  Control  group  attempted  to 
include  component  5)  Audio  or  6)  Video  in  their  lesson.  Of  the  four  remaining  technical  components,  the  Treatment 
group  scored  higher  on  average  in  every  component. 

H05:  There  will  be  no  statistically  significant  difference  in  the  degree  of  teacher  appeal  expressed  by  pre-service 
teachers  using  WE  + T-N-T  toward  developing  Web-based  lessons  than  from  pre-service  teachers  using  WE  only. 

H05  was  retained,  indicating  no  statistically  significant  difference  between  the  degree  of  positive  teacher 
appeal  from  pre-service  teachers  using  WE  + T-N-T  to  develop  Web-based  lessons  than  from  those  using  WE  only 
(t  = -0.6403,  p = 0.2617).  Data  were  compiled  and  analyzed  from  the  Computer  Attitude  Scale  (CAS).  The  four 
areas  associated  with  the  CAS  (low  anxiety,  high  confidence,  liking,  usefulness)  revealed  no  statistically  significant 
differences  between  the  two  groups.  The  results  can  be  found  in  Table  5. 


TableS.  Computer  Anxiety  Scale 

Group  Mean  Degree  of  Appeal  SD  df  p 


Treatment 

1 18.2450 

16.7980 

101 

0.2617 

(n  = 51) 

Control 

120.4807 

18.5723 

(n  = 52) 

Ho6:  There  will  be  no  statistically  significant  difference  between  the  likelihood  of  pre-service  teachers'  anticipation 
for  developing  Web-based  lessons  with  either  alternative  in  their  future  instructional  settings. 

H06  is  retained,  suggesting  no  statistically  significant  difference  in  the  anticipated  future  instructional 
development  of  Web-based  lessons  by  pre-service  teachers  using  WE  + T-N-T  to  develop  Web-based  lessons  or 
those  using  WE  only  (t  = -0.3946,  p = 0.3470).  Data  were  compiled  and  analyzed  from  the  Intended  Future  Use 
Questionnaire.  See  the  results  displayed  in  Table  6. 


Table  6.  Intended  Future  Use  of  Development  Alternative 


Group 

Mean  Score 

SD 

df 

P 

Treatment 
(n  = 51) 

8.4019 

1.9053 

94.3 

0.3470 

Control 
(n  = 52) 

8.5769 

2.5540 

Appeal  of  the  design/development  process  was  addressed  through  the  statistical  examination  of  H0s  and 
H06-  There  was  no  statistical  significance  in  the  differences  between  the  Control  group  and  the  Treatment  group  on 
H05  or  H06- 
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Talk-Aloud  Interviews 

The  post-hoc  talk-aloud  interviews  were  completed  by  telephone  with  students  from  both  the  Control  and 
Treatment  groups.  Two  students  were  selected  from  each  group  based  on  most  and  least  time  recorded  on  the  Work 
Time-Log.  Two  students  from  each  group  were  also  selected  based  on  the  highest  and  lowest  compiled  scores  of  the 
remaining  five  instruments  (Effort  Questionnaire,  Instructional  Components  Evaluation  Checklist,  Technical 
Components  Evaluation  Checklist,  Computer  Attitude  Scale,  Intended  Future  Use  Questionnaire).  In  theory,  eight 
talk-aloud  interviews  would  be  performed,  however,  one  of  the  individuals  identified  from  the  Control  group  fit  in 
two  categories  (least  time  spent  as  recorded  on  the  Work  Time-Log  and  high  composite  score  on  the  remaining  five 
instruments),  and  therefore  only  seven  participants  were  interviewed. 

Data  obtained  from  the  anecdotal  talk-aloud  interviews  revealed  that  one  participant  from  the  Treatment 
group  had  continued  using  the  T-N-T  template/shell  after  the  study  ended.  He  had  recently  completed  his  seventh 
lesson  in  only  20  minutes.  His  completion  time  for  the  initial  use  of  T-N-T  was  450  minutes.  Although  this  finding 
is  based  upon  a single  data  point,  the  substantial,  reduction  in  development  time  needs  to  be  explored  further  to 
determine  if  multiple  use  of  the  WE  + T-N-T  template  and  WE  only  results  in  similar  outcomes. 

Limitations 

Due  to  several  factors,  the  researcher  or  reader  would  be  vulnerable  in  drawing  direct  conclusions  from  the 
results  of  this  study.  The  following  limitations  of  this  study  should  be  considered  when  attempting  to  generalize 
from  the  findings  and/or  to  replicate  the  study. 

The  primary  limitation  of  this  study  was  the  failure  of  the  Control  group  to  develop  Web-based  lessons  that 
included  the  lesson  components  as  defined  within  this  study.  According  to  the  PTEP  methods  course  syllabus, 
lessons  components  include  goals,  objectives,  activities  and  performance  evaluation.  The  Control  group  did  not 
include  these  lesson  components  in  the  Web  sites  they  created.  Without  such  instructional  components,  the  nature  of 
the  sites  created  is  much  more  informational  rather  than  instructional/educational.  Had  the  Control  group  been 
required  to  rework  their  lessons  so  that  the  lessons  included  instructional  components,  the  results  of  the  comparisons 
between  the  Control  group  and  Treatment  group  in  regard  to  the  efficiency  (time  and  effort)  and  appeal  (low 
anxiety,  high  confidence,  liking,  usefulness  and  intended  future  use)  may  have  been  substantially  different. 

Restricting  this  study  to  a university  setting  is  the  greatest  strength  while  also  being  the  greatest  weakness. 
Template  design  and  interaction  of  pre-service  teachers  may  not  reflect  the  views  of  students  at  other  universities, 
smaller  or  larger  colleges,  and/or  community  colleges.  The  university  setting,  however,  offers  the  opportunity  to 
assess  students'  perspective  as  they  create  and  layout  their  lessons  using  the  template  format.  Ability  to  generalize  is 
largely  limited  due  to  the  sample  selection. 

A third  limitation  is  that  this  study  uses  a self-report.  Self-reports  are  susceptible  to  "error"  through  the 
difficulty  of  eliciting  honest,  accurate  responses.  In  this  study,  the  participants  are  not  under  pressure  to  produce  in 
order  to  be  paid;  nor  are  they  reporting  on  an  issue  that  will  influence  their  grade,  although  they  may  be  under  a 
certain  amount  of  time  constraint  pressure  because  of  their  own  organizational  capabilities.  The  assignment  will  be 
graded  "Pass"  or  "Fail"  based  upon  criteria  in  the  grading  rubric;  student  input  from  the  self-report  is  not  part  of  the 
grading  criteria.  These  factors  still  do  not  guarantee  the  truthfulness  of  the  responses.  However,  they  do  alleviate  the 
pressure  and  stress  to  answer  with  "teacher  pleasing"  responses. 

A final  limitation  of  this  study  is  control  for  experimenter  contamination.  The  intervention,  T-N-T  Web- 
based  lesson  template/shell,  was  designed  and  developed  by  the  researcher  and  unquestionably  there  is  researcher 
bias.  The  instruction  for  using  the  Web  editor  and  the  instruction  for  using  T-N-T  will  be  presented  by  the 
researcher.  Maintaining  fidelity  during  lesson  presentation  is  a limitation.  To  minimize  this  limitation,  adherence  to 
guidelines  for  each  scripted  lesson  will  be  described  in  Chapter  Three.  Another  aspect  of  experimenter 
contamination  is  that  data  collection  and  data  analysis  is  conducted  solely  by  the  researcher  and  subject  to  researcher 
bias.  To  minimize  this  limitation,  the  participants  complete  four  of  the  six  instruments.  Of  the  remaining  two 
instruments  one  is  scored  based  solely  upon  component  presence  and  the  other  is  scored  based  solely  upon 
component  functionality. 
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Description  of  the  T-N-T  Template/Shell 

Essentially,  the  lesson  template/she  11  is  a complete  Web-based  lesson  minus  the  content.  The  template/shell 
exists  as  a Web-based  resource/tool  for  the  teacher.  This  template/shell  is  designed  to  support  a pedagogically  sound 
linear  presentation  of  materials  and  activities  while  concomitantly  providing  a multiplicity  of  instructional 
scaffolding  cues  designed  to  guide  development  of  the  content.  Below  are  screen  captures  of  two  pages  from  the 
template. 
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Figure  1 . Screen  Capture  of  the  Navigation  Bar 


The  navigation  bar  includes  seven  links  (Goals,  Objectives,  Readings,  Activity,  Quiz,  Glossary  and 
Teacher).  Note  that  "Goals"  is  white  (inactive  link)  and  the  others  are  red  (active  links).  White  text  is  a visual  cue 
that  the  user  is  working  in  the  Goals  section.  The  title  "Lesson  Goals"  directly  below  the  navigation  bar  is  another 
indicator  of  location  and  page  purpose.  Scrolling  down  the  Lesson  Goals  page  reveals  a few  lines  of  direct 
instruction  to  the  developer. 

Each  page  of  the  template  (excluding  Glossary  and  Quiz)  includes  such  direct  instruction  and  a brief 
rationale  (theoretical  foundation)  for  the  suggested  approach  to  content  development  within  each  part  of  the  Web- 
based  lesson.  The  "Glossary"  and  "Teacher"  links  are  set  apart  within  the  navigation  bar.  This  segregation  is 
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intentional  as  glossary  and  teacher  sections  are  provided  as  Tools  for  the  teacher/designer  and  not  as  having  the 
same  instructional  rationale/purpose  as  the  other  five  components  of  the  lesson.  The  one  link  located  on  the 
navigation  bar  that  does  not  open  a Web  page  is  titled  "Teacher."  This  link  is  an  E-mail  link  to  the  instructor. 

The  lesson  template  also  includes  a ready-to-use  ten  question  multiple-choice  quiz.  The  interactive  "Quiz" 
is  a JavaScript  program  that  solicits  the  user  to  answer  questions  relevant  to  the  lesson  content  and  receive 
immediate  feedback  on  their  performance. 

When  providing  content  for  the  Quiz  section  of  the  lesson,  the  teacher/designer  is  prompted  to  insert 
questions  pertaining  to  the  lesson  and  potential  answers  (correct  and  incorrect).  The  feedback  section  of  the  Quiz  is  a 
program  designed  to  give  students  appropriate  information  about  their  performance.  Note  that  in  the  following 
screen  capture,  answers  to  the  various  questions  have  been  selected  and  the  user  has  clicked  on  the  "Calculate 
Results"  button. 

The  student  performance  feedback  section  of  the  Quiz  page  provides  the  student  two  sets  of  information. 
First,  it  furnishes  a text-based  assessment  of  their  performance  and  secondly  it  allows  the  student  to  see  which 
questions  they  answer  correctly  and  incorrectly. 
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Figure  2.  Screen  Capture  of  Performance  Feedback  section  of  the  Quiz  Page 
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Conclusions 

This  study  examined  the  efficiency  (time  and  effort),  effectiveness  (inclusion  of  specific  instructional 
components  and  functionality  of  specific  technical  components),  and  appeal  (anxiety,  confidence,  liking,  usefulness 
and  intended  future  use)  of  two  lesson  developmental  alternatives  (WE  only  and  WE  + T-N-T)  by  pre-service 
teachers.  Statistical  analysis  of  six  hypotheses  confirmed  that  developing  Web-based  lessons  using  WE  + T-N-T  was 
more  effective  than  using  WE  only.  The  analysis  also  revealed  that  neither  developmental  alternative  was  superior  in 
terms  of  efficiency  or  appeal. 

Using  a tool  such  a T-N-T  significantly  reduces  the  requirements  for  technical  expertise  in  the  development 
of  a Web-based  lesson.  T-N-T  also  provides  scaffolding  for  the  creation  of  pedagogically  sound  instructional 
environments  by  focusing  attention  of  the  designer  on  instructional  strategies  and  methodologies. 
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Professor  Ann  De  Vaney  and  A Good  Conversation 


Randall  G.  Nichols 
University  of  Akron 


Introduction 

The  first  time  I met  Professor  Ann  De  Vaney  (Becker),  she  listened  well  to  a paper  I presented  at  a 
conference.  She  listened  attentively,  and  doing  so  is  one  of  the  very  best  qualities  of  a good  academic.  So,  when 
given  the  chance  to  review  her  work  and  write  a paper  for  this  symposium,  I wondered  if  this  quality  of  active  or 
good  listening  is  somehow  present  in  her  written  works.  And  if  something  like  good  listening  is  evident,  how  does  it 
manifest  itself?  So  I acquired  written  works  from  throughout  her  career,  starting  with  her  dissertation  and  ending 
with  an  end-of-the  20th  century  work. 

My  goal  is  not  primarily  to  examine  De  Vaney’s  intended  ideas  so  much  as  it  is  simply  to  see  what  her 
writing  generally  reveals  in  terms  of  something  like  good  listening.  I don’t  want  to  take  the  works  solely  at  face 
value  but  to  look  into  them  to  see  if  I can  find  her  paying  close  attention  to  people — to  research  subjects,  to  subjects 
otherwise  addressed  in  the  works,  even  perhaps  to  readers  of  her  works.  So,  I gathered  and  read  the  ten  works  listed 
below.  They  were  available  from  my  own  library,  from  ERIC,  and  from  Dissertation  Abstracts. 

The  Works 

• Verbal  Ability  and  Visual  Verbal  Modes  of  Presentation  in  the  Acquisition  of  a Poetic  Concept  (1974) 

• “A  Model  for  Critical  Dialogue:  A Study  for  Enhancing  Student  Response  to  Visuals”  (1982,  with  Robert, 

Roberto,  Muffoletto) 

• “Reader  Theories,  Cognitive  Theories  and  Educational  Media  Research”  ( 1 985) 

• “Rules  of  Evidence”  (1990) 

• “A  Grammar  of  Educational  T elevision”  (1991) 

• “Square  One  T elevision  and  Gender”  (1991,  with  Alejandra  Elenes) 

• “Reading  Educational  Computer  Programs”  ( 1 993) 

• “Introduction:  Background  to  Channel  One”  (1994) 

• “Voices  of  the  Founders:  Early  Discourses  in  Educational  Technology”  (1996,  with  Rebecca  Butler) 

• “Can  and  Need  Educational  Technology  Become  a Postmodern  Enterprise?”  ( 1 998) 


Method 

At  first,  I wanted  to  see  if,  in  her  written  works,  professor  De  Vaney  “listens”  to  people  as  well  as  she  had 
listened  to  me  at  the  annual  Meeting  of  the  Association  of  Educational  Communications  and  Technology  in  1986. 
Of  course,  I knew  that  a typical  or  literal  meaning  for  “listen”  was  not  good  enough;  I was  reading  her  work.  After 
casting  about  for  the  concept  that  best  means  the  quality  I was  investigating,  and  after  considering  “hearing,” 
“listening,”  and  various  communication  models,  I decided  simply  to  look  for  instances  when,  in  writing,  she  appears 
to  attend  to  people’s  interests  in  some  way.  Attending  to  another  person  is  what  we  do  in  a good  face-to-face 
conversation.  We  pay  attention  to  people  and  respond  based,  partly  anyway,  on  their  interests,  intentions,  needs,  and 
so  on. 

This  line  of  thinking  led  me  to  try  to  understand  her  work  by  asking  a rather  straightforward  set  of 
questions.  I wanted  to  start  by  first  simply  asking  what  each  work  is  about,  but  early  on,  I realized  that  the  questions 
being  devised  were  emanating  from  the  kinds  of  post-structural,  reader,  and  reception  theories  Ann  uses  in  her  work. 
She  (1985)  says,  “Meaning  can  only  be  understood  as  what  the  individual  intends.  It  is,  therefore,  the  individual’s 
intentions  which  produce  the  specific  relations  of  differences  or  similarities...”  (p.  10).  So,  I ask  not  only  what  a 
piece  is  about  but  what  she  means  and/or  intends.  Why  is  she  writing  about  something?  This  question  necessitates 
examining  the  way  in  which  she  proceeds  with  her  work-her  methodology. 

Professor  De  Vaney  says  (1985),  too,  that  in  post-structural  theory  we  need  to  be  aware  that,  in  discourse, 
certain  audiences  are  included  and  “space  for  certain  viewers  [audiences],  is  excluded”  (p.  11).  So,  I ask  about 
whom,  in  terms  of  her  apparent  audiences,  she  includes  and/or  excludes. 

Further,  people  and  knowledge  are  continually  changing.  As  De  Vaney  says  (1985),  “knowledge  accrues” 
(p.  3).  So  I look  for  what,  if  anything,  changes  in  her  writing  over  time  and  in  terms  of  the  foregoing  quest 

So,  I attempt  to  address  each  work  by  asking  basic  questions: 

1 . What  is  the  work  about,  and  what  are  her  (and  any  co-author’s)  intent  and  methodology? 
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2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests? 

Analysis  of  the  Works 

Please  note  that  Professor  De  Vaney  has  co-authors  on  three  of  the  works  examined  here.  So,  in  the  case  of 
those  three  works,  the  reader  should  not  assume  that  what  emerges  or  what  I say  is  totally  attributable  to  Ann.  Of 
course,  a reader  shouldn’t  assume  that  anyway,  given  what  we  are  about  to  learn  from  Ann’s  work. 

• Verbal  Ability  and  Visual  Verbal  Modes  of  Presentation  in  the  Acquisition  of  a Poetic  Concept  (Becker,  Ann  De 
Vaney,  1974) 

1.  What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

This  first  work  is  Professor  De  Vaney’s  dissertation.  She  writes  that,  “The  main  objective  of  this  study  was 
to  determine  interaction  between  verbal  aptitude  and  visual-verbal  modes  of  presentation  in  the  teaching  of  a 
concept”  (p.  3)  so  as  to  help  media  designers  and  teachers. 

The  concept  is  “poetic  metaphor,”  and  the  modes  of  presentation  are:  printed  verbal,  printed  verbal  with 
still  pictures,  spoken  verbal,  and  spoken  verbal  with  still  pictures.  She  also  is  interested  in  the  interaction  of  these 
modes  with  subjects’  verbal  aptitudes.  Measures  were  taken  on  two  dependent  variables,  comprehension  and 
learning  retention. 

She  concludes  that,  “one  strong  result  emerged.  The  sound/still  picture  treatment  (that  is,  the  slide  tape 
presentation)  was  superior  to  print,  print/still  pictures,  and  sound”  (p.  70)  “especially  for  low  verbal  ability  students” 
(P-  71). 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

Her  audience  clearly  is  primarily  her  dissertation  committee.  Academics/researchers,  instructional 
designers,  and  teachers  are  a primary,  intended  audience.  Learners  are  included  but  only  as  research  subjects  and  not 
directly  as  an  audience.  Research  subjects  and  readers  are  treated  reasonably.  That  is,  she  writes  in  an  expository 
(empirical,  logical,  rational)  style. 

She  studies  the  verbal  abilities  of  people,  which  implies  attending  to  or  listening  to  people.  She  works 
closely  with  people  in  terms  of  learning  and  communicative  ability  (print  verbal  ability). 

The  study  is  experimental  and  quantitative,  which  indicates  that  it  is  less  attentive  to  the  subjects.  Given  the 
writing  style,  the  audience  and  subjects  are  held  at  arm’s  length,  so  their  best  interests  probably  are  not  being  served 
as  well  as  they  might. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

Of  course,  this  is  the  first  work  studied,  so  nothing  has  changed.  It  is  a typical,  rational,  empirical, 
expository,  third  person  study  and  writing  from  the  1970s.  It  is  what  is  or  was  expected  in  a dissertation. 

• “A  Model  for  Critical  Dialogue:  A Study  for  Enhancing  Student  Response  to  Visuals”  (with  Robert  Muffoletto, 
1982) 

1 . What  is  the  work  about,  and  what  are  her  (and  any  co-author’s)  intent  and  methodology? 

The  authors  try  “to  explore  the  effectiveness  of  a pedagogic  strategy  which  involved  the  development  of 
descriptive  tactics  and  individual  exploration  into  the  meanings  and  significance  of  photographic  images.  It  was 
hoped  that  the  study  would  reveal  methods  which  enhanced  student  responses  toward  photographs,  their  production, 
function  and  value  in  educational  and  culutral  [sic]  settings”  (p.  3). 

Students  took  photos  and  discussed  them,  guided  by  categories  provided  by  De  Vaney  and  Muffoletto:  judgment, 
formal  or  visual,  poetic,  and  political.  Each  week  for  six  weeks,  students  were  shown  a different  Diane  Arbus’  photo.  After  the 
first  and  last  showings  and  without  discussion,  each  student  wrote  responses  about  the  photos.  Student  replies  were  used  to  form 
perceptual  categories. 

“The  results  of  the  study  revealed  a number  of  interesting  considerations  for  teachers  of  media.  The  study  suggested 
that  by  structuring  the  psychological  learning  environment  conceptual  shifts  could  occur.  It  also  suggested  that  conceptualization 
in  one  activity  could  be  crossed  over  to  another  activity  which  becomes  an  important  consideration  for  educators  when  using 
media  in  their  curriculum.  The  study  also  suggested  that  the  development  of  a critical  dialogue  model  for  students 
might  enhance  verbal  responses  to  photographs”  (p.  8). 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

This  work  directly  addresses  attendees  of  the  Annual  Meeting  of  the  Association  for  Educational 
Communications  and  Technology:  educational  technology  researchers,  media  specialists,  graduate  students,  and  so 
on.  Other  education  researchers,  academics,  and  teachers  generally  are  a less  direct  audience.  Again,  students  are 
subjects  of  the  study  and,  in  the  usual  tradition,  are  not  directly  addressed  as  an  audience  or  as  potential  users  of  the 
results  of  the  study.  All  study  subjects  and  direct  or  potential  audience  members  are  treated  quite  reasonably.  As  is 
usually  the  case  with  academic  reports,  a certain  rhetorical  distance  from  the  reader  and  student/subject  is  present. 
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Given  that  Ann  is  interested  in  enhancing  student  conceptual  responses  to  photographs,  students  are  treated 
as  capable  learners,  thinkers,  and  conversationalists. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  Compared  to  the  first  work,  Dr.  De  Vaney  pays  greater  attention  to  university  students  and  to  how  they 
conceive  of  and  think  about  photographs.  She  works  directly  with  them  to  help  them  be  more  thoughtful.  She 
expects  learners  to  be  more  critical,  in  the  sense  that  they  are  rational.  However,  she  does  not  write  directly  for  or  to 
students.  The  work  is  about  them. 

• “Reader  Theories,  Cognitive  Theories  and  Educational  Media  Research”  (1985) 

1.  What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

This  paper  is,  in  part,  an  exposition  of  theories  that  have  conventionally  guided  research  in 
educational/instructional  technology.  The  paper  also  argues  that  the  professional  field  should  shift  to  structural 
and/or  post- structural  theories  and  investigations,  especially  in  that  structuralism  “focuses  on  human  acts... that 
involve  cultural  construction  and  the  ways  that  speech  acts  involve  sentences”  (p.  3).  Structuralism  is  interested  in 
the  meaning  one  intends  to  convey,  and  “knowledge  accrues  by  perception  of  meaning,  not  by  information 
processing... the  relationship  between  the  observer  and  the  observed  becomes  primary.  Knowledge. . .resides  in  the 
relationships  which  people  construct  and  then  perceive”  (p.  3).  “Meaning  or  understanding  is  generated  by  the 
learner  not  controlled  by  the  technology  or  its  designer”  (p.  6).  Post-structural  theories  go  farther,  emphasizing 
viewers  and  their  construction  of  meaning,  and  “meaning  can  only  be  understood  as  what  the  individual  intends”  (p. 
10). 

She  asks,  “How  would  the  learner  [reader,  student,  colleague,  writer]  be  perceived  in  post  structural 
studies?  Essentially,  he/she  would  be  understood  as  the  subject... of  the  meaning,  since  he/she  creates  meaning  from 
the  text  or  media  presentation. . .the  learner  is  viewed  as  a social  person  belonging  to  a group,  not  as  an  individual 
person  whose  brain  is  the  unit”  (p.  10).  So  group  construction  of  knowledge  is  studied.  Conversely,  messages  may 
be  encoded  so  that  “space  for  certain  viewers,  is  excluded.  Yet,  these  exclusions  may  not  be  obvious  to  the  designer. 
It  is  the  viewer  who  has  to  tell  us”  (p.  1 1). 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

This  work  directly  addresses  attendees  of  the  Annual  Meeting  of  the  Association  for  Educational 
Communications  and  Technology,  though  education  and  educational  technology  researchers,  academics,  theorists, 
and  designers  (and  perhaps  teachers)  are  likely  to  be  a secondary  audience,  given  the  venue  for  the  publication. 
Given  that  the  piece  is  written  in  a mostly  conventional  academic  fashion,  the  audience  is  treated  reasonably.  Here, 
too,  learners  are  never  directly  an  audience  for  her  work,  though  we  can  surmise  that  learners  are  meant  to  be  the 
ultimate  receivers  of  the  results  of  her  work. 

Even  though  the  work  is  somewhat  distanced  from  learners,  Ann’s  meaningful  attention  to  them  is  evident  in  this 
paper.  After  all,  she  views  the  learner  “as  a social  person  belonging  to  a group,  not  as  an  individual  person  whose 
brain  is  the  unit.” 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  Because  of  whom  she  is  addressing  directly,  she  is  attending  again  to  the  learner  at  a distance,  but  she 
writes  as  though  learners  have  agency  and  voice  and  importance. 

Here,  Ann  has  left  the  realm  of  empirical/quantitative  research  and  immerses  herself  in  naturalistic 
investigations  in  education.  But  she  goes  beyond  the  mundane  versions  of  most  forms  of  so-called  naturalistic 
inquiry  and  begins  to  show  attraction  to  what  I know  as  critical  theory,  which  is  concerned,  at  its  best  anyway,  with 
the  likes  of  human  freedom,  dignity,  and  social  standing. 

• “Rules  of  Evidence”  (1990) 

1.  What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

Ann  writes  that,  “I  will  discuss  whether  the  establishment  of  rules  of  evidence  is  possible  or  even  necessary.  An 
alternative  framing  of  the  question  about  rules  of  evidence  is,  ‘How  do  researchers  and  practitioners  in  a specific 
field  know  what  they  know?’  I will  describe  the  concept  of  evidence  in  law  and  science.  Specifically,  I will  show 
that  the  adoption  of  statistics  as  evidence  in  educational  research  was  historically  a clear  misreading  of  the  ‘scientific 
method’”  (p.  8). 

She  argues  that  education,  as  with  the  social  science  of  law,  is  based  in  context  and  language  and,  so, 
cannot  be  value  free  and  mechanistic.  Instead,  law  and  education  research  are  activities  in  which  evidence  is 
collected  and  judged  inductively  not  deductively.  She  goes  beyond  now  common  objections  to  applying  many  parts 
of  the  scientific  process  to  educational  research  to  point  out  that,  like  science,  “educational  researchers  are 
attempting  to  describe  and  measure  capricious,  discontinuous  and  not  easily  controlled  experiences”  (p.  16). 
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Among  other  conclusions,  she  says  that,  “the  conception  of  science  and  scientific  rigor  which  dominates 
the  area  of  educational  research,  and  which  was  borrowed  from  the  field  of  psychology,  is  overly  constrained  by  the 
adoption  of  narrow  methods  and  limited  use  of  the  mathematics  of  probability”  (p.  17).  She  wants  readers  to  expand 
our  understanding  of  educational  research. 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

Ann  writes  directly  for  educational  technology  researchers  who  try  to  expand  their  understandings  of 
educational  technology  and  for  scholars  of  diverse  backgrounds,  interests,  and  disciplines.  Readers  are  treated 
reasonably.  Again,  students  are  not  a direct  audience. 

She  attends  to  the  best  interests  of  research  participants  generally  by  arguing  for  a research  approach  that  is 
more  inclusive  of  and  context-bound  to  people’s  lives. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  The  writing  style  is  the  same  expository/academic  form  used  previously. 

To  the  extent  that  she  wants  researchers  to  be  more  inclusive  and  contextual,  she  is  closer  to  readers  and 
research  participants.  For  the  first  time,  a bit  of  affect  appears — when  she  says  readers  should  remain  “oppositional” 
not  only  in  terms  of  conventional  educational  research  but  (she  implies)  in  all  education  work.  The  argument  to 
broaden  research  paradigms  is  much  like  the  last  piece  reviewed  here,  though  this  piece  is  even  more  basic  in  that  it 
gets  at  a philosophic/rational  basis  for  post-structural  research. 

• “A  Grammar  of  Educational  Television”  (1991) 

1.  What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

This  piece  explains  a way  of  analyzing  how  students  read  educational  television  and  make  meaning  from  it. 
The  theory  section  of  the  chapter  “borrows  concepts  from  semiotics  and  suggests  a grammatical  analysis  of 
television.  Such  an  analysis  would  describe  units  of  television  construction  such  as  frame,  shot  and  sequence,  along 
syntagmatic  as  well  as  paradigmatic  lines”  (p.  255).  “In  conjunction  with  syntactical  analysis  which  examines  the 
actual  relationships  among  the  categories  of  construction,  paradigmatic  analysis.,  .would  provide  a description  of  the 
potential  relationships  between  and  among  the  units  of  television  construction”  (p.  256).  She  then  shows  how 
semiotics  has  been  “the  most  thorough  and  promising  description  of  the  language  of  media”  (p.  257)  and  applies  the 
description  to  TV,  ETV,  and  ITV. 

She  then  proposes  a model  for  the  method,  procedures,  and  data  analyses  of  the  grammar  of  television.  A 
panel  of  experts  views  a program  several  times,  generates  focus  questions,  and  identifies  program  formats.  Program 
segments  which  are  part  of  the  format  are  identified,  the  experts  record  segment  structures,  and  then  they  run  a 
reliability  check  on  these  data.  Then  data  are  appraised  for  program  segment  patterns.  Borrowed  codes  are  noted  and 
traced.  Social/cultural  origins  of  the  codes  are  traced,  and  the  social/cultural  content  and  meanings  are  examined. 

She  wants  to  give  researchers  a way  to  include  social/cultural  issues  when  studying  instructional  media 
because  often  researchers  have  abdicated  their  responsibility  to  do  so  (p.  276). 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

This  work  is  written  for  an  audience  of  educational  researchers  and  theorists.  She  treats  readers  very 
reasonably.  Here,  too,  she  does  not  speak  directly  to  students  (or  to  teachers). 

Not  all  Ann’s  work  exemplifies  the  qualities  of  good  listening/communication  that  I am  looking  for.  Some 
work  is  just  expository.  This  work  only  very  narrowly  explicates  how  students,  professors,  or  other  academics  do  or 
can  communicate/hear  one  another  well!  Ann  has  the  teachers’  and  students’  best  interest  at  heart  ultimately,  but  this 
piece  approaches  them  from  a distance. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  “A  Grammar  of  Educational  Television”  is  very  theoretical.  The  goal  of  thinking  through  and  proposing 
a model  by  which  educational  television  might  better  be  understood  is  very  lofty,  given  that  few  have  suggested 
such  a thing.  But  of  the  works  reviewed,  this  is  the  one  in  which  she  is  most  distant  from  the  people  I suspect  she  is 
trying  most  to  help  or  teach,  i.e.,  learners. 

• Square  One  Television  and  Gender  (with  Alejandra  Elenes.  1991) 

1.  What  is  the  work  about,  and  what  are  her  (and  any  co-author’s)  intent  and  methodology? 

The  authors  analyze  a children’s  TV  program  that  generally  is  accepted  as  positive  because  it  intends  to 
teach  what  many  of  us  agree  is  needed-math  skills.  The  authors  use  a form  of  post  structural  reader  theory  called 
“reception  theory”  to  understand  SOTV  in  terms  of  sexism  and  racism. 

They  describe  SOTV  as  “the  quintiessential  [sic]  post  modem  children’s  program,  since  it  employs  the 
latest  TV  technology  to  create  fractured  narratives  and  messages  which,  in  turn,  fragment  the  subject  positions  of  the 
viewers”  (p.  7).  Multiple  and  fast  changing  formats  are  used  to  gain  the  children’s  attention  about  math. 
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Unfortunately,  the  children  are  also  being  taught,  “how  a 12-  or  13-year-old  boy  and  girl  should  act,  dress  and  speak 
in  sexual  [even  pornographic]  fashion”  (p.  1 8).  Children  are  taught  that  people  with  non-American  accents  are  to  be 
parodied  (p.  19).  SOTV  signals  a return  to  racism,  albeit  a subtler  form  of  racism  (p.  20).  Children  are  subjected  to 
this  sexism  and  racism  in  20-40  percent  of  all  SOTV  programs. 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

This  work  directly  addresses  attendees  of  the  Annual  Meeting  of  the  Association  for  Educational 
Communications  and  Technology;  though  any  reader  would  likely  get  the  feeling  Ann  and  Alejandra  are  primarily 
talking  to  the  people  at  the  Children’s  Television  Workshop  (who  produce  and  design  SOTV),  directing  them  to 
change  their  programming.  Ostensibly,  though,  the  piece  is  written  for  educational  technologists,  researchers,  and 
designers.  These  audiences  are  treated  reasonably,  and  they  are  addressed  directly  and  strongly.  Ann  and  Alejandra 
do  not  directly  address  the  recipients  (children)  of  this  programming  that  is  shot-through  with  racism  and  sexism, 
though  we  know  it  is  for  the  children  that  they  are  exposing  the  program  biases. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  (and  Alejandra)  remains  concerned  with  helping  and  improving  the  field 
of  educational  technology. 

This  work  is  a cultural  critique,  and  the  authors  are  able  to  show  us  the  full  range  of  what  children  learn 
from  this  program.  Further,  the  tone  and  affect  of  this  piece  are  bolder  than  in  any  work  so  far.  It  has  a strong,  good 
moral  tone  because  it  involves  children  and  it  is  written  with  a strength  of  belief.  What  is  notable,  given  what  I set 
out  to  look  for  in  Ann’s  work,  is  when  she  says  that  her  “analysis  of  SOTV  is.,  .designed  to  investigate  the  way 
viewers  make  sense  of  and  learn  from  Sesame  Street  and  SOTV ’ (page  10)  (emphasis  added).  The  idea  of  “sense” 
indicates  she  wants  people,  especially  children,  to  understand  by  more  than  just  rational  means.  She  wants  people  to 
understand  TV  more  completely  by  using  a sense  of  cultural. 

• “Reading  Educational  Computer  Programs”  (in  Muffoletto  & Knupfer,  1993) 

1 . What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

Here,  Ann  uses  reader  theory  again  to  look  at  how  a computer  program  “makes  its  users  a subject  of  and 
subjected  to  particular  discourses  and  their  concomitant  values  and  knowledge  structures”  (p.  181).  Reception  theory 
sees  knowledge  as  a social  construct,  and  in  doing  so,  “it  can  uncover  the  manner  in  which  cultural  messages  are 
enfolded  in  the  rhetoric  of. ..an  educational  computer  program”  (p.  182).  The  theory  allows  Ann  “to  consider  the 
relationships  among  user,  program,  and  producer...”  (p.  82).  With  this  theory,  Ann  sets  out  to  uncover  cultural 
messages  in  Where  in  the  World  is  Carmen  San  Diego  ( WWCSD ),  the  computer  game. 

She  concludes  that  people  are  diminished  by  some  computer  programs  because  the  programs  treat  them  as 
logical  problems-solvers  who  value  immediate  feedback,  efficiency,  and  productivity,  who  are  not  frustrated  by  lack 
of  spontaneity  and  who  are  mechanized  subjects.  At  the  same  time  computer  users  are  people  who  must  participate 
in  larger  social  discourses  of  computing  whose  participants  produce  and  are  embedded  in  “a  range  of  aesthetic, 
moral,  and  political  value  judgments. . .produced  and  regulated  by  the  language  of  the  participants”(p.  183). 
Critically,  she  notes  that  a person  in  this  discourse  “may  be  totally  unaware  of  the  constructed  nature  of  these  value 
judgments  and  may  believe  them  to  be  natural  or  self-evident”  (p.  183). 

Further,  WWCSD  itself 

provides  an  invitation  to  users  to  participate  in  a rational-logical  discourse  that  is  conflated  with 
discourses  of  sexism  and  racism.  It  addresses  students  of  any  race  or  gender  in  a disrespectful 
manner.  Young  girls  are  not  only  invited  to  perceive  themselves  as  objects,  but  also  because  this 
game  is  sanctioned  as  an  “educational”  imprimatur,  they  are  encouraged  to  believe  that  this 
perception  is  appropriate.  Boys  fare  little  better,  for  the  program  empowers  them  in  chauvinistic 
ways  which  will  serve  them  poorly  in  their  daily  lives.... People  of  color  are  ignored  in  the  old 
versions  and  trivialized  in  the  1991  version.  This  treatment  means  that  the  majority  of  the 
California  school  population  is  eased  out  of  subjectivity,  unless  people  of  color  care  to  participate 
in  a racist  discourse”  (p.  191). 

In  order  to  subvert  these  conditions,  and  being  able  to  find  only  one  good  example  of  alternative 
instructional  software,  Ann  suggests  these  guidelines: 

• place  the  technology  in  the  hands  of  subjects  of  alternative  discourses  whose  rhetoric  includes  goals  of 
equity  and  personal  freedom 

• place  the  technology  in  the  hands  of  ethnic  groups  and  allow  them  to  represent  their  thinking  and 
behavior  in  an  authentic  manner 

• place  the  technology  in  the  hands  of  women  for  the  purposes  of  creating  cooperative  software  (p.  195). 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 
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The  audience  for  this  work  appears  to  be  educational  technologists  and  academics  generally  and,  perhaps, 
computer  program  designers,  all  of  whom  she  treats  reasonably.  Those  whom  the  programming  is  about  and  for — 
learners — are  treated  with  respect.  Learners  themselves,  recipients  of  the  critique  and  anything  resulting  from  it,  are 
not  addressed  directly.  Obviously,  though,  Ann  urges  that  producers  of  computer  programs  should  alter  their 
programming  so  that  users  are  treated  as  people  who  not  only  value  efficiency  and  the  like  but  who  also  are 
spontaneous  and  participate  in  a larger  social  context  of  aesthetics,  morals,  and  judgments.  In  this  fuller  treatment, 
Ann  is  respectful  of  leamers/users. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  In  this  piece,  she  is  speaking  whole-heartedly  about  the  relations  of  educational  computing  and  its 
production  of  sexism,  racism,  objectification,  over-rationalization.  She  urges  greater  self-determination,  freedom, 
and  democracy  in  technology  uses,  especially  for  children  and  women.  She  is  more  obviously  subversive  of  biased 
educational  thinking  and  materials  than  in  any  other  work  so  far. 

• “Reading  the  Ads:  The  Bacchanalian  Adolescence”  (De  Vaney,  1994) 

1 . What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

Ann  uses  reader  theory  again  to  analyze  Channel  One  ads  from  1990  and  1992.  She  asks,  “How  do  these 
ads  construct  their  subjects  or  position  their  viewers?  In  other  words,  just  whom  do  these  ads  think  their  viewers 
are?”  (p.  139) 

She  concludes  that  the  ads  borrow  from  commercial  TV  forms.  They  are  postmodern  in  that  they  feature 
“fractured  narratives,  fragmented  images,  heightened  use  of  jump  cuts,  excessive  use  of  dancing  and  singing,  and 
startling  juxtaposition  of  images”  (p.  145).  The  people  in  the  ads  are  almost  totally  white,  beautiful,  and  sexual.  The 
ads  focus  on  pleasure,  and  “no  labor  is  depicted”  (p.  146). 

She  does  not  want  to  suggest  that  teens  always  use  the  images  in  Channel  One  commercials  as  guidelines 
for  living,  but  she  does  suggest  that  “if  Hollywood  and  commercial  television  and  classroom  TV  present  only  a 
Bacchanalian  adolescence,  then  these  images  may  assist  teens  in  the  construction  of  adolescence  as  primarily  a 
party”  (p.  147).  In  these  ways,  the  ads  are  irresponsible  (p.  151). 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

The  primary  audience  appears  to  be  academics/researchers,  though  she  seems  also  to  writing  directly  to 
Chris  Whittle  and  Whittle  Communication— or  to  anyone  trying  to  influence  the  culture  and  commerce.  She  treats 
the  audience  reasonably.  Partly  because  of  where  it  appears,  in  a text  available  mostly  to  people  in  higher  education, 
she  is  not  writing  directly  to  teachers  or  students.  However,  we  can  see  in  the  dedication  in  whom  she  is  most 
interested:  “With  respect  and  love  for  the  thousands  of  elementary  and  secondary  students  we  have  taught.” 

Ironically,  if  teachers  in  the  primary  audience  take  this  piece  and/or  its  ideas  to  their  own  students,  who  are 
not  the  primary  intended  readers,  the  piece  may  have  more  effect  than  if  the  commercial  interests  read  it.  Students 
may  change,  whereas  commercial  interests  aren’t. 

3.  What  changes  in  her  work  and  in  her  attention  to  people’s  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  She  still  is  trying  to  change  the  way  media,  in  this  case  school/media  advertising,  produce  and/or 
influence  younger  leamers/viewers/users  while  ignoring  minority  groups  and  women.  Though  not  as  strongly,  the 
sense  of  urgency  felt  in  Square  One  Television  and  Gender  in  this  piece,  too. 

• “Voices  of  the  Founders:  Early  Discourses  in  Educational  Technology”  (with  Rebecca  P.  Butler.  In 
Jonassen,  1996) 

1 . What  is  the  work  about,  and  what  are  her  (and  any  co-author’s)  intent  and  methodology? 

In  this  work,  De  Vaney  (and  Butler)  derives  meaning  from  audiotapes  and  written  text  of  the  founders  of 
the  field  of  educational  technology.  She  uses  reader  theory  and  neo-theoretical  analysis  to  talk  about  the  voices  and 
texts  in  the  “amorphous  field”  of  educational  technology/audiovisual  instruction  in  roughly  the  first  half  of  the  20th 
century  in  America. 

She  concludes,  among  other  things,  that  “The  rhetoric  of  the  founders  and  those  who  follow  indicate  that 
the  basis  of  this  field  will  always  be  hardware,  with  its  concomitant  marketplace  and  governmental  interests.  If  we 
turn  away  from  this,  our  voices  will  remain  within  the  academy.  We  will  be  talking  to  ourselves.  Whatever 
tempering  influence  we  could  have  had  will  be  lost  “ (p.  38). 

She  devotes  quite  a bit  of  attention  to  women  in  the  field,  and  concludes  that  “Although  women  in 
educational  technology  became  more  prominent  [in  the  decades  after  WWII],  the  placement  of  women  in  this  field 
is  still  uneven  today....  today,  women  continue  working  towards  equal  recognition,  opportunity,  and  responsibility 
within  educational  technology”  (p.  43). 
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She  speaks  directly  to  a democracy  and  communication  when  she  points  out  that  the  field  today  “gathered  a 
berth  in  the  academy  with  the  discourses  represented  in  World  War  II  research,  operant  conditioning,  and  military 
training... Perhaps  the  audiovisual  educators...  no  ted  the  efficiency  and  effectiveness  of  military  pedagogy  operating 
particularly  in  the  service  of  democracy,  and  where  convinced  that  education  should  proceed  down  the  same  road” 
(p.  38). 

She  hears  the  people  involved  in  the  formation  of  the  field  and  speaks  of  the  ups  and  downs  of  the 
valorization  of  humanistic  discourses  and  democracy  and  the  common  person  associated  the  field  of  educational 
technology.  She  allows  the  voices  of  the  founders  to  be  heard.  She  speaks  of  the  human  condition  and  “One  of  the 
things  that  was  valued  in  WWII  was  the  preservation  of  democracy,  and  many  projects  were  conflated  with  that 
desire.  Unfortunately,  the  methods  for  accomplishing  that  preservation  at  that  moment  in  time  were  undemocratic, 

1. e.,  hierarchical  and  militaristic”  (p.  38).  She  also  notes  that  democratic  ideas  of  the  founders  included  a “model  of 
action”  (p.  38)  whereby  they  got  all  sorts  of  work  done. 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

This  piece  is  directly  for  researchers,  faculty,  and  students  in  educational  technology.  Ann  favors  women  in 
these  groups  getting  more  prominence  in  the  field  and  working  toward  more  recognition,  opportunity,  and 
responsibility.  She  literally  listens  quite  closely — and  well--to  the  founders.  Readers  and  audiences  are  treated 
reasonably  and  respectfully.  In  this  work,  she  and  Becky  Butler  give  others  voice  by  writing  about  others  and  by 
speaking  for  women,  founders,  democracy,  freedom,  and  equality. 

Though  we  know  Ann  cares  about  them,  here  too,  she  does  not  directly  speak  to  students. 

3.  What  changes  in  her  work  and  in  her  attention  to  people's  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology  and  the  places  of  women  and  minorities.  In  terms  of  its  methodology,  attention  to  women  in  the  field, 
and  relations  to  societal-cultural  issues,  this  is  an  unusual  history  of  the  field.  Readers  can  see  the  importance  of 
connections  with  politics,  hardware,  and  prior  economic  issues.  She  displays  a great  deal  of  fairness  toward 
“fathers”  in  the  field  of  educational  technology;  she  realizes  the  culture  in  which  the  founders  existed  and  how  that 
affected  them  and  the  profession. 

• “Can  and  Need  Educational  Technology  Become  a Postmodern  Enterprise?”  (1998) 

1 . What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

This  piece  takes  a pointed  look  at  historical,  hegemonic  features  of  educational  philosophy:  “European  [sic] 
and  Americans  writing  about  democracy  also  voiced  their  values  and  views  of  humans  while  calling  for  equality  for 
everyone.  Yet,  their  epistemological  assumptions  sketched  the  human  as  a rational/logical  entity  that  actually 
reflected  the  person  of  European  male.  Thinking  they  were  acultural,  as  educational  psychologists  and  technologists 
thought  they  were  acultural,  these  writers  called  for  equality  for  the  white  male....  The  postmodern  critique  of 
subjectivity  uncovers  this  fallacy”  (p.  78).  De  Vaney  describes  the  development  of  educational  technology  as  a 
modem  project,  with  its  belief  in  progress,  profits,  patriotism,  machines,  and  human  equity.  So,  educational 
technologists  often  have  exhibited  a lack  of  caring  about  gender,  power,  race,  and  human  equity.  Effects  of 
modernity  in  education  include  the  commodification  of  students  and,  at  best,  an  interesting  construction  of 
subjectivity.  Learners  became  classroom  capital  to  be  sold  to  commercial  enterprises.  Learners  became  mental 
models  stripped  of  social  and  cultural  dimensions;  notions  of  human  diversity  were  constrained. 

Today,  many  people  in  the  field  of  educational  technology  resist  the  unavoidable  changes  brought  on  by  a 
postmodern  position.  Postmodernists  study  race,  gender,  ethnicity,  and  power  relations.  They  are  skeptics  who  often 
attack/deconstruct  modernity  and  the  beliefs  in  which  educational  technology  is  embedded.  They  often  deconstruct 
by  examining  discourse.  Teachers/technologists  are  unaware  of  the  ways  in  which  their  discourses  are  not  only 
created  by  them  but  enslave  them  to  narrow  views  of  human  diversity. 

However,  Ann  says,  “postmodern  discussions  of  subjectivity  allow  teachers  to  discern  how  students  are 
constructed  by  instructional  media  and  software”  (p.  77).  She  says  that, 

Alternatively,  videos  and  software  must  not  only  picture  girls  and  children  of  color  but 
also  provide  respectful  representations  of  diverse  groups.  If  in  a video  a credible  Muslim  boy  is 
presented  with  respect  and  given  agency,  there  would  then  be  room  for  the  formation  of  a new 
subjectivity  in  the  classroom.  Knowledge  about  subjectivity  allows  teachers  to  resist  subtly 
stereotypic  constructions  as  well  as  to  recognize  when  equitable  representations  and  those  with 
agency  come  along  (p.  77). 

She  is  interested  in  resisting  the  commodification  of  students  and  the  capitalization  of  classrooms. 
Postmodernist  positions  help  teachers  to  resist.  She  is  interested  in  respect  for  people. 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 
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She  speaks  directly  to  the  readers  of  Theory  Into  Practice.  The  readership  includes  teachers  and  teacher 
educators.  These  audiences  are  treated  reasonably.  She  also  treats  readers  as  people  with  agency,  as  she  encourages 
them  to  resist  problematic  discourses/media.  She  listens  and  speaks  to  teachers  and  tries  to  help  them.  She  respects 
them  in  this  discourse.  Once  more,  she  does  not  speak  directly  to  students,  though  we  know  that  ultimately  it  is  they 
as  well  as  their  teachers  for  whom  she  is  writing. 

3.  What  changes  in  her  work  and  in  her  attention  to  peoples’  interests  over  time? 

The  topic  is  different,  though  Ann  remains  concerned  with  helping  and  improving  the  field  of  educational 
technology.  In  the  greatest  detail  so  far,  she  addresses  distinctions  between  modem  and  postmodern  and  how  they 
relate  to  cultural  considerations  in  education,  educational  media,  classroom  practices,  teachers,  and  formation  of 
student  subjectivities.  She  is  interested  in  resisting  the  objectifying  and  commodifying  of  students  so  that  they  may 
develop  greater  agency  and  self.  She  notes  again  that  resistance  is  needed — resistance  by  teachers  and,  one  suspects, 
resistance  by  students. 

Results 

The  following  general  answers  to  the  guiding  questions  emerge  from  the  analysis  of  the  ten  written  works 

studied. 

1.  What  is  the  work  about,  and  what  are  her  intent  and  methodology? 

Dr.  De  Vaney  writes  about  ten  different  topics  in  the  field  of  educational  technology,  topics  which  include 
aptitude-treatment-interaction  and  media,  teaching  critical  responses  to  photographs,  a field  shift  to  post-structural 
and  reader  reception  theories,  rules  that  constitute  knowing  in  the  field,  how  to  read  the  structures  of  educational 
television,  sexism  and  racism  in  ETV  programming,  the  diminishing  of  children  by  TV  programs,  degradation  of  the 
subject  by  TV  ads,  a post-structural  history  reading  of  the  field  and  the  struggle  by  women  in  that  field,  and  the  need 
for  a postmodern  philosophy  of  educational  technology.  In  general,  the  work  is  about  laudable  goals:  human 
education,  equality,  and  voice. 

She  often  uses  her  writing  to  improve  and/or  defend  the  lives  of  learners.  She  intends  to  teach  those  in 
educational  technology  that  they/we  should  be  interested  in  more  culturally  relevant  forms  of  work  that  may  resist 
the  diminishing  of  learners  that  older  forms  often  promote.  She  is  especially  concerned  with  improving  the  lives  of 
women  and  minorities.  She  wants  to  subvert  the  conventional  paradigms  of  educational  technology  field.  She  wants 
to  make  our  work  more  supportive  of  democracy. 

Her  research  methodology  broadens  quickly  in  her  career  to  sociological/cultural  models  of  research.  In 
most  of  her  works,  she  uses  qualitative,  post-structural  (often,  reader  reception  theory),  and  inductive  forms  to 
examine  the  topics.  She  uses  expository  writing  to  report  her  findings.  I mean  “expository”  to  be  the  use  of  precise 
and  logical  written  statements  and  explanations. 

2.  Who  is  her  audience,  and  how  does  she  attend  to  the  interests  of  those  she  addresses? 

In  many  of  the  works  examined  here,  the  primary  audience  is  AECT  convention  goers:  researchers, 
professors,  graduate  students,  and  perhaps  some  school  media  specialists.  Otherwise,  her  audience  is  educational 
technology  researchers,  theorists,  and  academics  generally.  She  speaks  to  women  and  minorities  related  to  these 
fields.  She  speaks  to  Chris  Whittle  of  Channel  One  and  the  Edison  Project  and  to  designers  and  producers  at 
Children’s  Television  Workshop  and  SOTV.  Only  occasionally  do  teachers  appear  to  be  the  intended  audience.  She 
virtually  never  directly  addresses  students  in  her  work — except  as  participants  in  studies. 

All  readers  are  treated  reasonably  and  respectfully — even  those  who  would  use  technology  to  diminish 
learners.  Even  when  her  writing  approaches  ire  toward  these  diminishes,  she  is  reasonable  and  respectful.  Most 
significantly  from  the  point  of  view  of  communicating/listening/responding,  Ann’s  work  strives  (though  indirectly 
in  some  cases)  to  give  people  (researchers,  women,  teachers,  students,  readers  generally)  greater  voice,  agency,  and 
knowledge.  She  is  dedicated  to  gaining  greater  voice,  freedom,  democracy,  and  respect  for  many  people. 

3.  What  changes  in  her  work  and  in  her  attention  to  peoples’  interests  over  time? 

Of  course,  the  specific  topics  of  her  works  change  from  writing  to  writing,  though  they  stay  within  the  field 
of  educational  technology.  After  a very  short  time,  her  work  is  more  foundational  and  philosophic  in  that  it  seeks 
wisdom  as  it  deals  with  very  basic  issues.  The  pieces  get  stronger-especially  when  she  talks  about  youngsters--and 
address  democracy,  freedom,  respect,  and  voice.  The  work  is  somewhat  more  subversive  over  time;  it  asks  readers 
to  resist  convention  and  change  the  cultural  makeup  of  the  field. 
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Discussion 

Ironically,  to  well  and  fully  carry  out  an  investigation  such  as  the  present  one,  I should  have  given  her  more 
voice;  I should  have  spoken  directly  with  Dr.  De  Vaney  about  these  questions  and  issues.  As  she  says,  “Meaning  or 
understanding  is  generated  by  the  learner  not  controlled  by  the  technology  [written  word]  or  its  designer”  (p.  6). 

In  terms  of  the  listening  I have  been  looking  for,  Ann’s  writing  is  sometimes  a very  collaborative  process  in 
which  she  works  directly  with  other  authors  (e.g.,  Muffelotto,  Elenes,  and  Butler,  herein).  To  this  basic  extent,  she 
communicates  well. 

But  from  within  her  work,  too,  she  carries  on  a conversation  or,  to  use  her  language,  a “discourse”  with 
many  people.  Discourse  is  not  a concept  constrained  to  oral  or  even  written  language.  As  De  Vaney  and  Butler 
(1996)  say,  “We  consider  discourses.  ..to  be  texts  ‘writ  large ’(p.  5)  “...discourses  are  invisible  systems  of  thought” 
(P-  5)- 

It  is  clear  in  her  works  that  Ann  respectful  and  careful  of  learners;  I refer  you  to  the  dedication  she  makes  in 
Watching  Channel  One:  With  respect  and  love  for  the  thousands  of  elementary  and  secondary  students  we  have 
taught.  This  is  evidence  that  she  is  concerned  with  learners’  best  interests  and  that,  in  this  way,  she  is  carrying  on  a 
good  discourse  with  them. 

Her  discourses  with  students  and  educational  technology  broaden  and  deepen  over  time;  at  least  if  we  look 
at  the  first  and  last  works  studied  here.  The  dissertation  is  very  empirical  and  rather  narrowly  focused,  but  the  last 
piece,  “Can  and  Need  Educational  Technology  Become  a Postmodern  Enterprise?”  is  sweeping  in  scope  and 
concerned  with  very  basic  issues  of  human  education. 

Ann’s  work  encourages  the  possibility  of  school  democracy  and,  therefore,  I believe,  true  school  reform. 
Remember,  she  says, 

• place  the  technology  in  the  hands  of  subjects  of  alternative  discourses  whose  rhetoric  includes  goals  of 
equity  and  personal  freedom 

• place  the  technology  in  the  hands  of  ethnic  groups  and  allow  them  to  represent  their  thinking  and 
behavior  in  an  authentic  manner 

• place  the  technology  in  the  hands  of  women  for  the  purposes  of  creating  cooperative  software  (1993,  p. 
195). 

This  concern  with  democracy  and  local  control  of  technology  is  evidence  of  her  attending  to  and  having 
good  discourse  with  people.  It  is  also  unusual  in  that,  as  far  as  I can  see,  virtually  no  democratic  schooling  takes 
place  in  America. 

Her  work  attempts  to  be  fair,  which  is  an  attribute  I would  claim  is  essential  for  good  discourse.  In  her 
attention  to  and  sensibilities  for  helping  children,  women,  and  minorities,  she  cannot  be  accused  of  being  unfair  to 
males — as  is  evidenced  by  her  treatment  of  the  founders  who  may  have  made  some  questionable  choices  when  they 
invested  in  technology,  given  its  bent  toward  the  undemocratic  (De  Vaney  and  Butler,  1996). 

With  this  concern  for  what  democracy  can  and  should  be,  Ann  keeps  a sense  of  a good  future  in  mind 
without  negating  or  forgetting  the  past.  This,  among  other  attributes,  makes  her  writing  moral,  which  is  another 
attribute  of  good  discourse/conversation. 

Her  work  is  a reliable,  valid  analysis  of  educational  technology  in  one  form  or  another,  as  evidenced  partly 
by  her  successful  career  and,  mostly,  by  her  own  “rules  of  evidence.” 

So,  the  quality  of  attending  to  my  thoughts  that  I first  noticed  in  Ann  is  present  in  her  written  work,  too. 
However,  I do  not  want  to  give  the  impression  that  I agree  with  everything  she  says.  For  example,  I am  not  sure  that 
too  many  postmodernists  and  modernists  are  too  polemical,  as  she  says  in  “Can  and  Need  Educational  Technology 
Become  a Postmodern  Enterprise?”  Sometimes  an  argumentative  appeal  can  bring  democracy  more  quickly  than  a 
more  purely  reasonable  appeal. 

My  only  major  question  about  her  scholarship  is  that  she  virtually  never  directly  speaks  to  the  students  (and 
sometimes  the  teachers)  for  whom  she  writes!  If  Ann  were  a better  communicator,  wouldn’t  she  also  have  spoken  or 
written  directly  to  learners  more  than  she  has?  Wouldn’t  she  have  used  a more  direct,  inclusive  assessment  of 
people’s  comprehension  of  poetic  metaphor  or  teachers’  need  to  resist  commodification? 

Conclusion 

Those  of  us  in  educational  technology  would  do  well  to  consider  following  the  paths  she  has  gone  down,  be 
they  her  research  topics,  methodologies,  and  philosophies;  be  they  the  people  about  whom  she  is  interested;  or  be 
they  the  ethical  stance  she  takes. 

I will  say  that  examining  her  work  has  made  me  realize  that  I virtually  never  do  my  work  directly  for 
students — which  now  seems  to  me  to  be  a very  serious  error.  Writing  for  and  speaking  to  academics-including 
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teachers,  principals,  media  specialists,  and  professors — is  not  likely  to  do  any  good  for  students  in  schools.  Of 
course  almost  no  one  writes  or  speaks  for  learners.  But  I know  of  almost  no  professors  who  write  directly  to  students 
of  any  age  to  encourage  them  to  work  toward  self-voice,  agency,  democracy  or  even  learning!  Maybe  we  should 
write  books  (or  whatever)  for  students  so  that  they  will  understand  and  make  decisions  about  educational  technology 
for  themselves  instead  of  being  told  what  to  do  all  the  time! 

It  is  clear  that  in  her  written  work  Ann  is  doing  well  at  working  toward  understanding  how  people 
communicate,  how  they  make  meaning  with  one  another.  However,  throughout  this  paper  I have  been  uncertain 
about  what  I mean  by  Ann’s  ability  to  listen  or  communicate.  My  shifting  use  of  terms  indicates  my  uncertainty. 
Perhaps  I can  be  clearest  if  I tell  you  that  my  examination  of  her  written  work  leads  me  to  conclude,  mostly,  that 
Ann  is  very  good  at  carrying  on  the  kind  of  good  conversation  Jane  Roland  Martin  describes: 

A good  conversation  is  neither  a fight  nor  a contest.  Circular  in  form,  cooperative  in  manner,  and 
constructive  in  intent,  it  is  an  interchange  of  ideas  by  those  who  see  themselves  not  as  adversaries 
but  as  human  beings  come  together  to  talk  and  listen  and  learn  from  one  another. 

People  have  a lot  to  learn  from  Ann. 
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Faculty  Point  Of  View  Of  A Faculty  Development  Program 


H.  Ferhan  Odabasi 
Coskun  Bayrak 
Anadolu  University,  Turkey 


Background 

The  importance  of  professional  development  of  teachers  in  higher  education  has  been  a rising  issue  in  the 
recent  years.  In  fact  the  developments  in  education  reveal  that  so  much  as  the  knowledge  reflected,  the  higher 
education  instutions  should  be  learning  organizations  themselves,  as  well  (Latchem  and  Lockwood,  1998).  Being 
professional  as  a higher  education  teacher  involves  a knowledge  base  in  both  the  subject  or  dicipline  area  and  in 
education,  thus  all  higher  education  teachers  must  learn  to  create  a synthesis  between  their  knowledge  of  their 
discipline  and  how  students  learn  (Beaty,  1998).  Professional  development  activities,  in  HE  realized  to  improve 
teaching,  enhance  student  learning  and  facilitate  faculty  research  and  other  scholar  activities,  ty  to  over  come  this 
“double  professionalism”  (Beaty,  1998).  Anadolu  University  in  Turkey,  which  carries  the  unique  distance  education 
in  HE  in  the  country  and  known  as  a pioneer  institution  in  many  different  areas,  started  a program  called  “Educating 
the  Educators”  in  1999.  The  program  is  aimed  to  assist  the  faculty  to  enhance  their  teaching  skills.  Conscious  of  that 
many  of  the  faculty  have  had  no  or  little  experience  in  teaching  skills,  the  university  wanted  to  start  the  program 
with  a general  teacher  education  program  approach. 

The  Purpose  And  Methodology  Of  The  Study 

This  study  was  carried  out  in  Anadolu  University,  Turkey.  The  population  of  the  study  consisted  of  78  full 
time  faculty  as  assistant  professors. 

Table  1 Population  of  the  Study 


Gender 

f 

% 

Female 

38 

49 

Male 

40 

51 

Field  of  Study 

Social  Sciences 

37 

47 

Applied  Sciences 

16 

21 

Fine  Arts 

7 

9 

Technical  Sciences 

6 

8 

Health  Sciences 

7 

9 

Communication  Arts 

5 

6 

Adult  Learning  Experience 

Yes 

21 

27 

No 

57 

73 

The  questionnaire  used  in  the  study  was  designed  so  as  to  get  the  necessary  responses  for  the  evaluation  of 
the  program. 

The  questionnaire  consisted  of  three  parts.  The  first  part  elicited  the  personal  information  The  second  part 
consisted  of  questions  to  evaluate  the  given  courses  and  the  last,  third  tried  to  elicit  suggestions  about  how  the  future 
programs  should  be  designed. 

The  purpose  of  the  study  was  to  learn  the  opinions  of  the  faculty  about  a faculty  development  program 
carried  out  and  their  suggestions  for  a future  program. 


Results 

Since  the  results  of  the  first  part  of  the  questionnaire  are  summarized  in  Table  1,  only  the  second  and  third 
parts  will  be  discussed  here. 
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Evaluation  of  the  Courses  by  the  Participants 

Altogether  there  were  nine  courses  offered  in  the  program.  The  participants  were  asked  to  evaluate  the  courses  as 
from  the  point  of  view  of  nine  criteria  as; 

Efficient  use  of  time 
Satisfactory  content 

Applicability  of  the  knowledge  presented 
Level  of  meeting  the  needs 
Encouraging  participation 
Solving  teaching  problems 
Clearity  of  the  subject 
Enviromental  Appropriateness 
Sufficient  practice  and  case  studies 

The  analysis  of  the  results  revealed  that  the  courses  ranked,  as  from  the  most  productive  to  the  least,  as  follows  ; 

1.  Faculty-Student  Relations 

2.  Characteristics  of  the  Adolesence 

3.  Use  of  Technological  Resources 

4.  Educational  Technology 

5.  Class  Management 

6.  Academic  Counsellig 

7.  Instructional  Methods  and  Strategies 

8.  Planning  the  Instruction 

9.  Evaluating  the  Instruction 

Faculty’s  Suggestions  for  a Future  Program 

In  the  third  part  of  the  questionnaire  the  participants  were  first  asked  to  name  the  courses  that  should  take 
part  in  a future  program  and  the  results  were  as  follows; 


COURSES 

f 

Effective  Speaking  &Listening 

62 

Use  of  Technological  Resources 

62 

Faculty  & Student  Relations 

60 

Academic  Counselling 

60 

Effective  Presentation 

60 

Characterictics  of  Adolesence 

55 

Educational  Technology 

53 

Academic  Progress 

49 

Evaluating  the  Instruction 

48 

Class  Management 

47 

Academic  Regulations 

32 

Instructional  Methods  &Strategies 

29 

Organization  and  Administration  of  University 

27 

Creative  Drama 

23 

Planning  the  Instruction 

23 

The  second  item  of  the  third  part  of  the  questionnaire  asked  the  priorities  that  a future  program  should  carry 
and  the  results  were  as  follows; 


PRIORITIES  OF  A FACULTY  DEVELOPMENT 

f 

PROGRAM 

Needs  Analysis 

26 

Efficient  Curriculum 

23 
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Voluntary  Participation 

18 

Effective  Timing 

11 

Classification  According  to  Area  of  Study 

10 

Informing  the  Participants  Beforehand 

8 

Small  Group  Interaction 

4 

Conclusion 

The  results  of  the  study  revealed  how  the  faculty  who  attented  the  program  evaluated  the  program  and 
made  suggestions  about  a future  program.  The  results  of  the  evaluation  of  the  courses  showed  that  the  faculty  mostly 
found  the  Faculty- Student  Relations  course  to  be  most  beneficial.  However  Planning  & Evaluating  The  Instruction 
courses  which  are  more  teaching  skill  oriented  came  at  last.  This  might  be  a clue  that  the  faculty  rather  need  courses 
that  support  teaching  but  not  right  in  the  middle  of  it.  This  fact  is  also  revealed  in  the  faculty’s  course  suggestions 
for  a future  program.  Here  the  faculty  suggest  Effective  Speaking  & Listening;  Faculty  & Student  Relations  and 
Academic  Counselling  Courses  in  the  highest  rank  for  future  programs 

When  it  comes  to  the  priorities  that  a future  program  should  carry;  the  faculty  reveal  that  a Needs  Analysis  is  the  top 
priority.  Then  comes  the  Efficient  Curriculum.  This  result  shows  the  weakness  of  the  program. 
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Sixth  Graders’  Motivation  During  Problem-Based  Learning 


Susan  Pedersen 
Texas  A&M  University 


Abstract 

This  study  employed  a mixed  methods  design  to  examine  students  ’ motivation  during  PBL,  comparing  it  to 
their  motivation  during  typical  class  activities.  Analysis  of  the  quantitative  data  shows  that  students  demonstrate 
more  intrinsically  motivated  behavior  during  PBL  than  during  their  regular  class  activities.  The  qualitative  data 
suggest  that  the  greater  opportunity  for  collaboration  and  student  control  over  class  activities  afforded  by  PBL  may 
partially  account  for  students  ' enhanced  motivation. 

The  literature  on  intrinsic  motivation  enumerates  the  many  ways  it  can  benefit  learning.  Intrinsically 
motivated  students  persist  in  the  face  of  failure,  undertake  challenging  aspects  of  a task,  show  creativity,  and  remain 
cognitively  engaged  in  tasks  longer  than  their  extrinsically  motivated  peers  (Ormrod,  1995).  Given  these  conditions, 
it  is  no  wonder  that  interest  in  learning  for  its  own  sake  has  been  shown  to  be  positively  correlated  with  achievement 
(Alexander  & Murphy,  1998;  Schiefele,  1996). 

How  do  we  account  for  variations  in  motivational  orientation?  Research  conducted  within  a developmental 
framework  (e.g.  Harter,  1978)  cast  intrinsic  versus  extrinsic  orientation  as  a property  of  the  individual,  an  enduring 
characteristic  that  is  shaped  over  time  by  successes  and  failures  (Schunk,  2000).  Similarly,  models  that  have  arisen 
out  of  a cognitive  framework  have  primarily  examined  learners’  perceptions  and  beliefs,  tacitly  defining 
motivational  orientation  as  a product  of  the  covert  thought  processes  of  the  individual  (Bong,  1996).  In  both  cases 
the  effect  of  situational  variables  on  motivation  has  largely  been  overlooked. 

Recent  theories  of  motivation  have  taken  a different  view,  using  a "learner  in  context"  approach  in  the 
study  of  motivation.  In  their  theory  of  situated  motivation,  Paris  and  Turner  (1995)  have  postulated  that  motivation, 
like  cognition,  is  situated.  They  argue  that  an  individual’s  motivational  beliefs  and  behaviors  are  derived  from 
contextual  transactions.  From  this  perspective,  motivation  is  necessarily  unstable,  varying  with  each  context 
because  the  context  itself  causes  variations  in  learners'  perceptions  and  goals. 

This  interest  in  the  role  of  context  has  led  researchers  to  examine  the  types  of  approaches  and  the 
conditions  within  learning  environments  that  can  encourage  learners  to  assume  an  intrinsic  motivational  orientation 
to  learning.  One  approach  that  holds  promise  is  problem-based  learning  (PBL).  In  PBL,  all  learning  occurs  as  a 
result  of  students'  efforts  to  solve  a complex  problem.  Instruction  begins  with  the  presentation  of  the  problem;  as 
students  grapple  with  the  problem,  they  realize  that  they  lack  information  and  skills  that  they  need  to  develop  a 
solution.  Students  then  engage  in  self-directed  study  to  meet  these  learning  needs.  As  a result,  activity  is  largely 
under  student  control  and  learning  is  meaningful  because  students  are  the  ones  who  have  generated  a need  for  it. 

PBL  is  widely  regarded  as  highly  motivating  to  students.  Barrows  and  Tamblyn  (1980)  suggest  that  students  are 
motivated  in  PBL  by  the  internal  rewards  of  learning,  and  not  by  grades  or  external  rewards.  Hmelo  and  Ferrari 
(1997)  point  out  that  PBL  calls  for  an  action  on  the  part  of  the  student  (the  development  of  a solution)  and  that 
students  are  often  more  motivated  by  the  pragmatic  goals  involved  in  knowledge  building  for  action  then  in 
knowledge  building  for  its  own  sake.  To  the  extent  that  students  see  the  relevance  of  the  problems  used  during  PBL 
to  their  own  lives,  this  approach  may  tap  into  personal  interest.  This  is  certainly  the  case  in  medical  education 
where  students  are  typically  challenged  to  deal  with  the  same  types  of  situations  they  will  encounter  in  their  chosen 
profession. 

Research  findings  suggest  that  PBL  may  help  to  promote  an  intrinsic  motivational  orientation  in  students. 
Moore-West,  Harrington,  Mennin,  Kaufman,  and  Skipper  (1989)  found  that  PBL  students  gave  higher  ratings  of 
their  experience  than  non-PBL  students  on  a variety  of  topics,  including  meaningfulness,  emotional  climate,  and 
student  interactions.  Medical  students  enrolled  in  PBL  programs  found  their  education  more  relevant  to  their  future 
careers  than  their  traditional  track  peers  (West,  Umbland,  & Lucero,  1985).  PBL  students  were  more  likely  to 
describe  their  preclinical  program  as  difficult,  engaging,  and  useful,  while  non-PBL  students  were  more  likely  to 
describe  theirs  as  irrelevant,  passive,  and  boring  (Albanese  & Mitchell,  1993).  In  a longitudinal  study  of  the  effects 
of  a course  taught  entirely  through  PBL,  graduates  were  asked  during  telephone  interviews  about  favorite  courses, 
and  about  the  course  that  was  most  helpful  in  teaching  them  problem  finding,  problem  solving,  critical  thinking,  and 
ethical  decision  making.  Students  overwhelmingly  favored  the  class  taught  exclusively  through  PBL  on  all  of  these 
questions  (Stepien,  Gallagher,  and  Workman,  1993). 
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Taken  together,  these  findings  suggest  that  students  enjoy  the  challenge  of  PBL,  that  they  find  topics  more 
interesting  when  they  encounter  them  through  PBL,  and  that  they  feel  this  approach  leads  to  more  profound  learning 
than  other  approaches.  In  this  way,  PBL  may  enhance  learners1  motivation,  encouraging  them  to  behave  in  a more 
intrinsically  motivated  manner  than  they  do  during  the  teacher-centered  learning  activities  they  typically  encounter 
in  school. 

The  theory  of  situated  motivation  may  help  to  account  for  why  students  find  PBL  environments  motivating. 
Paris  and  Turner  (1995)  identify  four  characteristics  of  contexts  that  may  support  an  intrinsic  orientation  in  learners, 
all  of  which  are  incorporated  in  PBL  environments.  The  first,  choice , implies  the  ability  to  choose  among 
alternative  courses  of  action,  and  reflects  the  personal  interest  of  the  individual.  Learners  who  are  free  to  choose  the 
activity  that  they  find  most  interesting  and  productive  are  more  likely  to  be  thoughtfully  engaged  in  a task  and  show 
persistence  and  self-regulation.  PBL  environments  provide  some  constraints  for  choosing  (students  must  do  work 
that  contributes  to  a solution  plan)  yet  they  are  rich  enough  to  permit  multiple  paths  through  the  problem  space.  As 
learners  identify  possible  courses  of  action,  they  are  free  to  pursue  those  they  want,  and  may  change  course  as  they 
think  best.  The  second  characteristic,  challenge , contributes  to  motivation  by  encouraging  risk-taking,  and 
successful  performance  on  challenging  tasks  enhances  the  individual’s  cognitive  self-assessments  of  their 
competence  and  efficacy.  The  problems  used  in  PBL  are  selected  because  they  are  both  challenging  and 
manageable  for  students.  Control  has  been  shown  to  promote  motivation  by  increasing  learner  interest,  confidence, 
and  sense  of  self-worth.  The  extended  periods  of  self-directed  study  and  the  level  of  autonomy  students  enjoy 
during  PBL  offer  them  a high  degree  of  control  over  their  learning  activities.  Finally,  collaboration  impacts 
motivation  in  several  ways.  Their  peers’  statements  and  interests  may  stimulate  learners’  interests,  encouraging 
them  to  pursue  information  they  had  previously  overlooked.  Students  may  be  more  likely  to  try  out  certain  activities 
when  their  peers  have  modeled  it,  and  peer  modeling  may  offer  students  a means  by  which  to  monitor  their  own 
level  of  accomplishment.  Collaboration  may  also  foster  a sense  of  responsibility  to  others  that  motivates  learners  to 
persist  and  perform  up  to  their  potential.  PBL’s  use  of  collaboration  therefore  can  be  seen  as  supporting  both 
student  construction  of  knowledge  and  motivation. 

Paris  and  Turner's  theory  of  situated  motivation  has  another  implication  for  the  examination  of  motivation. 
They  stress  that  motivation  is  neither  a characteristic  of  the  learner  or  a property  of  an  event,  but  rather  the  result  of 
the  interaction  of  the  two.  From  this  perspective,  a learner’s  motivational  orientation  in  a given  situation  should  be 
the  product  of  both  his  or  her  enduring  motivational  beliefs  and  the  impact  of  the  features  of  that  situation  on  those 
beliefs.  If  motivation  is  a product  of  a learner  by  context  interaction,  then  a difference  in  motivational  orientation 
should  be  accompanied  by  a correlation  between  motivational  levels  in  different  environments.  This  correlation 
would  be  the  result  of  some  degree  of  stability  in  learners'  motivational  orientations.  For  example,  a group  of 
learners  may  express  a greater  preference  for  challenge  under  some  conditions  than  others,  but  learners  who 
generally  prefer  challenge  should  rate  preference  for  challenge  more  highly  in  all  situations  than  students  who 
generally  prefer  easy  work. 

Purpose  of  the  Study 

The  purpose  of  this  study  was  threefold.  First,  this  study  sought  to  expand  the  literature  on  problem-based 
learning  by  examining  its  impact  on  the  motivational  orientation  of  middle  school  students.  PBL  has  been  used 
extensively  with  mature  learners  (medical  school,  graduate  programs)  and  the  gifted.  However,  only  limited 
information  on  its  effectiveness  with  younger,  regular  education  learners  has  appeared  in  research  literature. 

Second,  PBL's  potential  impact  on  motivation  makes  it  a prime  candidate  through  which  to  test  the  implication  of 
Paris  and  Turner's  theory  that  motivation  is  the  product  of  a learner  by  context  interaction.  Evidence  supporting  this 
theory  would  show  that  learners  differ  by  context  in  their  motivational  orientation,  but  that  their  scores  in  different 
contexts  are  correlated.  This  correlation  would  indicate  the  role  that  the  enduring  characteristics  of  learners  play  in 
motivation,  while  differences  in  scores  could  be  attributed  to  contextual  factors. 

Finally,  using  the  theory  of  situated  motivation  as  a starting  point,  this  study  also  sought  to  begin  an  investigation 
into  the  aspects  of  PBL  environments  that  may  help  to  enhance  motivation.  By  examining  reactions  to  the  learning 
environment  used  in  this  study,  it  may  be  possible  to  discern  some  of  the  characteristics  of  PBL  that  enhance  student 
motivation. 

Three  research  questions  were  posed:  v 

1.  Do  learners  report  different  levels  of  intrinsic  motivation  for  regular  science  class  environments  and  this  PBL 
environment? 

2.  Do  the  levels  of  intrinsic  motivation  students  show  for  regular  science  class  predict  the  levels  they  show  for  this 
PBL  environment? 

3.  What  aspects  of  this  PBL  environment  enhance  student  motivation? 
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Method 


Participants 

Students  in  three  intact  sixth  grade  science  classes  at  a suburban  middle  school  in  the  southwestern  United 
States  participated  in  the  study,  with  N=66.  The  same  teacher  taught  all  three  classes.  The  scores  of  four  students 
were  discarded.  One  of  these  students  transferred  out  of  the  school  before  the  completion  of  the  study.  Three  other 
students  possessed  limited  English  proficiency,  and  normally  received  a modified  curriculum.  The  classroom 
teacher  and  an  ESL  teacher  worked  extensively  with  these  students,  often  directing  their  work.  Because  students* 
ownership  of  the  problem  solving  process  is  a key  feature  of  PBL  environments  and  this  feature  may  contribute  to 
the  positive  impact  PBL  has  on  learners*  motivation,  this  level  of  teacher  directiveness  might  have  interfered  with 
the  treatment.  Data  from  these  three  participants  were  discarded  because  they  may  not  accurately  reflect  the  impact 
of  PBL  on  motivation. 

The  ethnic  makeup  of  the  classes  reflected  the  school  at  large.  77%  of  the  participants  were  Caucasian, 
15%  were  Hispanic,  5%  were  African  American,  and  3%  were  Asian.  The  classes  were  split  almost  evenly  by 
gender,  with  two  more  girls  than  boys  participating  in  the  study. 

Treatment  Variable 

All  participants  completed  the  activities  in  Alien  Rescue,  a hypermedia  delivered  problem-based  learning 
environment  for  use  in  sixth  grade  science  classes.  The  science  fiction  premise  of  this  program  places  students  in 
the  role  of  young  scientists  aboard  a newly  operational  international  space  station  where  they  are  part  of  a 
worldwide  effort  to  rescue  alien  life  forms.  To  accomplish  this  goal,  students  learn  about  the  planets  and  large 
moons  of  our  solar  system  by  searching  existing  databases  and  designing  probes  to  gather  additional  information. 
While  the  primary  learning  objectives  of  Alien  Rescue  focus  on  astronomy  and  space  travel,  the  program  offers  ties 
to  other  areas  of  the  curriculum,  including  life  science  and  mathematics.  All  information  needed  to  solve  the 
problem  is  contained  within  the  virtual  environment  created  by  the  program,  though  it  is  structured  in  such  a way  as 
to  not  suggest  its  usefulness  or  lead  students  toward  a particular  solution.  The  program  normally  takes  fifteen  forty- 
five  minute  periods  to  complete. 

Data  Sources 

The  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom 

The  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom  (Harter,  1981)  was  administered  both 
before  and  after  students  participated  in  using  the  software  program.  During  the  pretest,  students  were  asked  to  think 
about  their  experiences  in  their  regular  science  classes  up  to  that  point.  During  the  posttest,  they  were  asked  to 
reflect  on  their  experiences  with  Alien  Rescue  and  imagine  participating  in  similar  projects  in  the  future.  The 
purpose  of  changing  the  instructions  was  to  determine  if  students  express  different  levels  of  intrinsic  motivation  in 
the  two  different  settings. 

The  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom  comprises  five  subscales,  though  only 
four  were  used  in  this  study.  Table  1 lists  the  subscales  used  and  provides  information  about  each.  The  scale  uses  a 
structured  alternative  format.  Learners  are  presented  with  two  “types**  of  kids,  then  asked  to  decide  which  type  they 
more  closely  resemble.  Once  they  have  chosen  the  type,  students  must  then  decide  if  that  choice  is  “really  true  for 
me"  or  "sort  of  true  for  me."  Items  are  scored  on  a 4-point  scale,  with  1 representing  the  maximum  extrinsic 
orientation  and  4 representing  the  maximum  intrinsic  orientation. 
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Table  1:  Subscales  in  the  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom 


Subscale 

Description 

Sample  Statement 

Subscale 

Reliability 

Preference  for 
Challenge  vs. 
Preference  for  Easy 
Work  assigned 

Does  the  student  like  hard, 
challenging  work  or  prefer 
easier  assignments? 

Some  kids  like  difficult 
problems  because  they 
enjoy  trying  to  figure  them 
out,  but  other  kids  don’t 
like  to  figure  out  difficult 
problems. 

.78  to  .84 

Curiosity/Interest 
vs.  Pleasing  the 
Teacher/Getting 
Good  Grades 

Does  the  student  work  to 
satisfy  his  or  her  own 
interest,  or  to  satisfy  the 
teacher  and  obtain  good 
grades? 

Some  kids  read  things 
because  they  are  interested 
in  the  subject  but  other 
kids  read  things  because 
the  teacher  wants  them  to. 

.68  to  .82 

Independent 
Mastery  vs. 
Dependence  on  the 
Teacher 

Does  the  child  prefer  to  do 
his/her  own  work  and 
figure  out  problems  on  his 
or  her  own,  or  does  the 
child  rely  on  the  teacher 
for  help  and  guidance  in 
these  areas? 

If  some  kids  get  stuck  on  a 
problem  they  ask  the 
teacher  for  help  but  other 
kids  keep  trying  to  figure 
out  the  problem  on  their 
own. 

.70  to  .78 

Independent 
Judgment  vs. 
Reliance  on  the 
Teacher’s  Judgment 

Does  the  child  feel  he/she 
is  capable  of  making 
judgments  about  what  to 
do,  or  does  he  or  she 
depend  on  the  teacher’s 
opinion  about  what  to  do? 

Some  kids  think  it’s  best  if 
they  decide  when  to  work 
on  each  school  subject  but 
other  kids  think  that  the 
teacher  is  the  best  one  to 
decide  when  to  work  on 
things. 

.75  to  .83 

Interviews 

Brief  interviews  with  students  were  conducted  at  the  end  of  the  unit,  after  students  had  completed  the 
attitude  questionnaire.  The  purpose  of  these  interviews  was  to  gather  additional  data  about  learners’  opinions  of 
learning  in  the  Alien  Rescue  environment  and  possible  reasons  this  environment  encourages  intrinsic  motivation. 

Students  were  interviewed  in  groups  of  two  and  three.  These  interviews  were  partially  structured.  Seven 
core  questions  were  asked  of  all  interviewees,  then  other  questions  were  added  to  encourage  students  to  elaborate  on 
the  opinions  they  expressed.  The  core  questions  were 
What  did  you  think  of  Alien  Rescuel 
What  did  you  like  most  about  Alien  Rescue ? 

Do  you  think  you  learned  any  science  during  Alien  Rescue ? 

Did  you  feel  you  had  control  over  what  you  did  during  Alien  Rescue ? Did  you  like  that?  Would  you  have 
liked  Alien  Rescue  as  much  if  you  hadn’t? 

How  did  you  feel  about  working  with  your  classmates?  Did  it  help? 

Did  you  find  the  problem  challenging? 

Would  you  want  to  do  programs  like  Alien  Rescue  in  the  future? 

Procedures 

The  study  was  conducted  during  students'  regular  science  class  times,  which  were  daily  45-minute  periods. 
Students  were  engaged  in  the  program  for  approximately  thirteen  class  periods,  with  an  additional  four  class  periods 
used  for  introductions,  data-collection,  and  a debriefing  session. 

Each  student  worked  at  his  or  her  own  computer.  Students  were  encouraged  to  work  together  to  share  the 
information  they  discovered  and  to  ask  each  other  for  help.  They  were  not,  however,  assigned  partners  or  told  they 
had  to  divide  up  the  work  in  any  way.  Daily  whole  class  meetings  were  held  at  the  beginning  of  the  period  so  that 
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students  could  share  their  findings  and  questions,  model  their  processes,  and  discuss  their  strategies  for  developing  a 
solution.  The  researcher  facilitated  these  discussions,  but  did  not  direct  student  activity. 

The  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom  was  administered  on  the  first  day  and 
the  fifteenth  day  of  the  study;  because  of  the  way  the  days  of  the  study  fell,  administrations  were  exactly  three  weeks 
apart.  Interviews  were  conducted  in  the  last  two  days  of  the  study. 

Data  Analysis 

Students’  pre  and  post  treatment  scores  on  the  Harter  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the 
Classroom  were  used  to  address  the  first  two  research  questions.  For  the  first  research  question,  “Do  learners  report 
different  levels  of  intrinsic  motivation  for  regular  science  class  environments  and  this  environment?"  the  data  were 
analyzed  using  a repeated  measures  analysis  of  variance  with  one  repeated  variable  (pre/post).  To  answer  the 
second  research  question,  “Do  the  levels  of  intrinsic  motivation  students  show  for  regular  science  class  predict  the 
levels  they  show  for  this  PBL  environment?”  a regression  analysis  was  conducted,  with  the  posttest  scores  as  the 
dependent  variable  and  the  pretest  scores  as  the  independent  variable.  The  resulting  R squared  was  used  to 
determine  if  a significant  portion  of  learners’  scores  on  the  posttest  could  be  predicted  by  their  pretest  scores.  In  this 
way,  it  was  possible  to  reflect  on  how  much  of  learners’  motivational  patterns  represent  enduring  characteristics  of 
the  learners,  and  how  much  of  that  variance  is  attributable  to  other  factors  including  the  type  of  environment. 

The  interviews  conducted  with  learners  at  the  conclusion  of  the  study  were  transcribed  and  analyzed  using 
a two-level  scheme  following  the  guidelines  provided  by  Miles  and  Huberman  (1994).  The  data  were  examined  for 
insights  into  the  causes  for  enhanced  motivational  orientation.  At  the  first  level,  codes  were  generated  using  the  four 
characteristics  suggested  by  the  theory  of  situated  motivation  (choice,  challenge,  control,  and  collaboration)  and  by 
multiple  passes  through  the  data.  At  the  second  level,  these  codes  were  regrouped  into  more  general  categories. 
The  data  were  sorted  into  categories  and  subcategories  according  to  their  common  themes.  Data  were  analyzed  by 
two  reviewers  and  disagreements  between  them  were  discussed  until  a .9  interrater  reliability  was  established. 

Results 

Quantitative  Data 

The  pre  and  post  test  scores  of  the  participants  were  compared  to  determine  if  learners  expressed  different 
levels  of  intrinsic  motivation  for  their  regular  science  class  environments  and  the  PBL  environment  offered  in 
programs  like  Alien  Rescue . Results  of  the  repeated  measures  analysis  showed  that  students’  scores  on  the  posttests 
were  significantly  higher  than  on  the  pretest  for  all  four  subscales,  as  shown  in  Table  2.  In  other  words,  students 
reported  a significantly  more  intrinsic  orientation  for  the  Alien  Rescue  environment  than  for  their  typical  class 
activities. 

Table  2:  Mean  Scores  and  Standard  Deviations  (in  Parentheses)  by  Treatment  Condition  on  Subscales  of  the  Scale 
of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom 


Subscale 

Pre  Treatment 
Means  and  SD 

Post  Treatment 
Means  and  SD 

F 

P 

Challenge 

2.58 

2.94 

21.5 

<.01 

(.77) 

(.58) 

Curiosity 

2.53 

2.79 

5.1 

<05 

(.95) 

(.48) 

Mastery 

2.65 

2.90 

13.40 

<.001 

(.69) 

(.51) 

Judgment 

2.61 

2.84 

12.47 

<.001 

m 

(57)  

To  answer  the  second  research  question,  a regression  analysis  was  conducted  to  determine  if  students’  pre 
treatment  scores  predicted  their  post  treatment  scores,  and  if  so,  to  calculate  the  amount  of  variance  in  the  post 
treatment  scores  that  could  be  explained  by  enduring  characteristics  of  the  learners.  The  results  are  shown  in  Table 
3.  In  three  of  the  subscales,  learners’  scores  on  the  pre  and  post  treatment  administrations  showed  a significant 
correlation.  In  the  challenge  subscale,  39*>o  of  the  variance  in  the  posttest  could  be  explained  by  students’  scores  on 
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the  pretest.  For  the  mastery  and  judgment  subscales,  these  figures  are  37%  and  36%  respectively.  These  results 
suggest  that  there  are  enduring  motivational  characteristics  of  students  that  help  to  account  for  the  level  of  intrinsic 
motivation  they  report  for  novel  learning  environments  like  the  one  offered  in  Alien  Rescue. 


Table  3:  Regression  Analysis  of  Pre  and  Posttests  on  Subscales  of  the  Scale  of  Intrinsic  versus  Extrinsic 
Orientation  in  the  Classroom 


Subscale 

t 

R squared 

Challenge 

6.44** 

.39 

Curiosity 

1.22 

- 

Mastery 

6.18** 

.37 

Judgment 

6.00** 

.36 

**significant  atp  < .01 


Qualitative  Data 

The  qualitative  data  were  examined  in  order  to  gain  a more  robust  understanding  of  students’  opinions  of 
the  Alien  Rescue  environment,  and  how  these  opinions  influenced  their  level  of  intrinsic  motivation.  Four  themes 
were  chosen  prior  to  the  interviews  through  an  examination  of  the  theory  of  situated  motivation.  These  themes  were 
choice,  challenge,  control,  and  collaboration.  Three  other  themes  emerged  from  the  analysis.  They  were  perceived 
educational  value,  computer  delivery,  and  novelty. 

Choice 

All  of  the  students  interviewed  indicated  that  they  would  like  to  work  on  other  problems  like  the  one 
presented  in  Alien  Rescue . When  asked  if  they  would  want  to  do  programs  like  this  all  the  time,  students’  responses 
ranged  from  “Not  really  all  the  time,  but  once  in  a while”  to  “most  of  the  time”  and  “Yeah,  like  everyday  after 
school  I run  home  and  get  on  the  computer.”  A few  students  added  stipulations  about  how  they  would  like  to  work, 
with  one  student  saying  he  would  only  want  to  do  programs  like  this  at  school,  not  at  home.  Another  student 
commented,  “I’d  rather  do  it  more  often  if  it  was  with  a group  than  by  myself.  Like  if  Alien  Rescue  was  for  sale  and 
I bought  it,  I would  want  to  have  three  computers  and  do  it  with  some  friends.” 

Challenge 

Most  of  the  students  interviewed  said  that  they  felt  Alien  Rescue  was  challenging,  but  not  overly  so,  that  “it 
wasn’t  too  hard,  it  wasn’t  too  easy.”  When  asked  how  they  liked  being  challenged,  students  responded  positively. 
“Yeah,  it  was  pretty  complicated.  I like  that.  I like  complicated  things.”  One  student  described  it  as  “fun 
challenging.” 

A few  students  noted  that  their  classes  rarely  presented  them  with  situations  as  challenging  as  the  one  in 
Alien  Rescue . One  student  explained  why  he  felt  Alien  Rescue  was  better  at  encouraging  him  to  tackle  challenges 
than  typical  school  activities: 

This  is  just  like  using  my  own  steps.  If  I can  solve  it  that  way  I would  try  to,  instead  of 
teachers  always  telling  you  the  right  way  and  I always  end  up  taking  the  easy  road  instead  of  at 
least  trying  to  do  something  different. 


Control 

All  of  the  students  interviewed  felt  that  they  had  control  over  their  actions  during  Alien  Rescue,  and  liked 
this  aspect  of  the  program.  When  asked  if  they  would  have  liked  to  have  more  direction  from  the  teacher,  most 
students  gave  a resounding  “No,”  and  said  that  the  program  would  not  have  been  as  much  fun  if  the  teacher  had 
taken  more  control  over  their  activities.  One  student  explained, 

Cause  if  you  want  to  make  a probe  you  want  to  make  a probe.  If  you  want  to  do  research, 
you  do  research.  But  it  wouldn’t  be  much  fun  if  you  had  to  make  a probe  or  you  had  to  do 
research. 

The  fact  that  students  had  control  over  their  own  process  meant  that  different  students  were  working  on 
different  aspects  of  the  problem  simultaneously.  One  student  saw  this  as  supportive  of  collaboration  and  a 
distribution  of  labor:  “if  everyone’s  doing  the  same  thing  you  can’t  find  different  things  out.” 

Two  students  commented  that  this  level  of  control  set  Alien  Rescue  apart  from  typical  school  activities: 


At  first  we  didn’t  know  what  to  do  so  we  just  looked  around,  but  it  was  real  fun  to  do  it  by 
ourselves,  cause  during  regular  classes  all  you  have  to  do  all  year  is  the  teacher  telling  you  what  to 
do,  but  she  said  we  could  do  what  we  wanted  to  do,  so  it  was  what  we  supposed  to  be  doing.  It 
was  really  fun  cause  we  got  to  choose  what  we  were  going  to  do. 

Probably  one  of  the  things  I liked  the  best  about  it  is  you  always  have  a teacher  telling  you  how  to 
do  something,  but  since  you’re  trying  to  find  the  problem  and  trying  to  solve  it,  you  get  to  solve  it 
your  own  way,  not  the  way  that  the  teacher  wants  you  to  do  it. 

Collaboration 

Students  in  all  three  treatment  conditions  said  that  they  felt  Alien  Rescue  provided  them  with  ample 
opportunities  to  collaborate  with  their  classmates,  and  that  they  both  gave  and  received  help.  Several  students  cited 
collaboration  as  one  of  their  favorite  aspects  of  Alien  Rescue.  Comments  included  “It  made  it  fun  and  challenging  at 
the  same  time,”  “The  funnest  part  was  working  around  with  a whole  bunch  of  people”  and  “I  liked  it  cause  it  just 
made  us  come  together  more  and  learn  thinking  skills  together.” 

The  opportunity  to  collaborate  led  to  a division  of  labor  for  many  students,  and  there 
were  several  comments  that  this  helped  them  to  work  more  quickly.  One  student  explained,  “We 
just  started  splitting  up  planets,  working  on  the  same  things,  saying  ‘What’s  your  hypothesis  so 
far?’  You  can  cover  more  space  with  two  people,  more  than  one.”  Another  student  explained 
how  this  sharing  of  responsibilities  worked  for  himself  and  two  of  his  peers: 

Me,  this  other  guy  (I  forgot  his  name)  and  Andey,  we  worked  together.  It’s  like  I’m  here,  he’s 
there,  and  he’s  there,  and  we  were  working  together.  I would  go  to  the  solar  system  and  I would 
look  up  planets  for  them  and  he  would  look  at  the  aliens  and  he  would  say  if  the  planet  would 
work  and  he  would  say,  “Okay,  it  needs  this  much  atmosphere.”  And  I would  look  for  a planet 
with  that  much  atmosphere  but  like  with  no  earthquakes. 

Collaboration  also  led  students  to  challenge  each  other’s  ideas,  making  it  possible  to  avoid  mistakes  and 
make  better  decisions  than  they  would  working  alone: 

Yeah,  and  you  could  find  out  their  hypotheses  about  that,  and  you  could  say  “Well,  that’s  not  my 
hypothesis,  so  how  did  you  come  up  with  that?”  and  then  they  can  tell  you  stuff  that  you  didn’t 
really  know.  You  could  compare  the  hypotheses.  Like  if  I didn’t  have  a piece  of  information  that 
he  did,  then  I would  just  send  them  to  a totally  wrong  planet. 

One  impediment  to  collaboration  was  discussed  by  a group  of  students.  Many  of  the  students  did  not  know 
each  other  well,  and  in  some  cases  did  not  even  know  their  classmates’  names.  Sixth  grade  is  the  first  grade  of 
middle  school,  and  this  study  was  conducted  in  November.  Ten  weeks  of  school  had  been  insufficient  for  students 
to  become  acquainted,  and  some  students  did  not  feel  comfortable  talking  to  peers  they  did  not  know.  Three 
students  who  had  collaborated  extensively  explained  that  they  worked  together  because  they  had  gone  to  the  same 
elementary  school  and  had  been  friends  for  several  years.  In  contrast,  they  did  not  work  with  another  student  seated 
next  to  them:  “Like  Jason,  he  sits  on  the  other  side  of  me,  and  was  hard  for  me  to  get  started  to  communicate  with 
him  because  I didn’t  know  him.”  Another  student  commented  that  before  Alien  Rescue  she  had  not  known  the 
classmate  seated  next  to  her  very  well,  but  that  during  the  course  of  the  program  they  had  become  “very  close” 
because  of  their  collaborative  efforts. 

Several  students  commented  that  they  rarely  had  the  opportunity  to  collaborate  in  their  regular  classes,  and 
when  asked,  all  of  these  students  said  they  preferred  to  work  together  as  they  had  during  Alien  Rescue.  Most  said 
that  it  made  learning  more  fun.  One  student  explained  that  the  freedom  of  movement  students  had  during  Alien 
Rescue  meant  less  reliance  on  the  teacher:  “Sometimes  we’re  sitting  next  to  someone  and  they  don’t  know  the 
answer,  and  I can’t  just  get  up  and  ask  someone,  so  I always  just  have  to  resort  to  a teacher.  I don’t  really  like  to  do 
that  sometimes.” 
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Perceived  Educational  Value 

Students  generally  felt  they  learned  a great  deal  from  their  experience  with  Alien  Rescue.  When  asked 
what  they  felt  they  had  learned,  most  students  identified  specific  scientific  facts  about  the  solar  system  and  the 
design  of  probes,  as  well  as  concepts  like  why  some  worlds  have  magnetic  field. 

In  addition  to  learning  science  content,  a few  students  talked  about  developing  problem-solving  skills.  Two 
students  remarked, 

It  teaches  you  a lot  about  how  to  make  steps  and  about  the  scientific  method.  If  you  just  skip 
around  and  not  go  step  by  step  it  will  usually  come  out  wrong  and  you  won’t  get  all  the 
information  you  need. 

Yeah,  and  how  you  went  through  a lot  of  steps.  First  you  have  to  do  research,  then  you  make  a 
hypothesis,  then  check  it,  and  if  it  was  wrong  you  have  to  go  over  it  again  and  study  all  the  aliens 
and  the  solar  system  and  everything  that  they  need  to  live  in. 

Several  students  said  they  felt  like  Alien  Rescue  gave  them  the  opportunity  to  work  like  scientists, 
identifying  hypotheses,  deciding  how  to  gather  the  information  they  needed,  and  collaborating  with  their  peers.  One 
student  commented, 

Normally  like  in  movies  and  stuff  you  think  of  scientists,  how  they  talk  together,  they  actually 
come  together  in  meetings  and  announce  everything  they  learned.  And  we  did  that  before  every 
class  period.  Really  and  we  could  get  up  out  of  our  seats,  so  actually  it  does  feel  like  scientists. 

Computers 

Two  other  themes  emerged  from  the  data  that  suggest  factors  other  than  the  characteristics  of  PBL  may 
have  contributed  to  the  more  intrinsic  orientation  students  expressed  for  the  Alien  Rescue  environment.  The  first  is 
the  impact  of  computers  in  general  or  this  computer-based  program  in  particular.  The  interactive  nature  of 
computers  themselves  may  encourage  students  to  explore,  taking  a mastery  orientation  and  enjoying  the  challenge  of 
figuring  out  the  functionality  of  a program.  Several  students  commented  that  their  textbooks  were  often  boring,  and 
that  being  able  to  work  on  the  computer  and  collaborate  with  their  friends  was  more  fun. 

Some  features  of  Alien  Rescue  may  also  encourage  students  to  take  an  intrinsic  orientation.  While  PBL 
encourages  exploration  by  giving  students  the  responsibility  for  generating  learning  needs  and  finding  the  resources 
to  meet  those  needs,  this  software  may  also  encourage  exploration  through  the  use  of  rich  media  and  a science 
fiction  premise.  When  asked  what  they  liked  best  about  the  program,  most  students  responded  by  citing  their 
favorite  feature  of  the  program  or  commenting  on  the  problem  itself,  not  by  describing  aspects  of  the  way  they 
worked.  Describing  the  alien  computer  component  of  the  program,  which  contains  QTVR  movies  of  the  aliens  and 
their  habitats,  one  student  commented,  "It  was  kinda  cool  when  you  could  just  look  at  all  the  aliens,  and  turn  them 
around  and  everything.  Then  you  could  look  at  their  habitats  and  their  solar  system."  In  describing  the  problem, 
another  student  said,  "I  thought  the  whole  idea  of  the  aliens  coming  and  us  having  to  find  a place  for  them  and  being 
able  to  be  on  their  computer  is  pretty  neat."  Several  students  also  described  the  program  as  a game.  One  student 
explained,  "I  think  of  it  as  a game  because  it's  on  computers,"  indicating  that  both  the  program  and  its  delivery 
medium  may  suggest  play.  Rieber  (1996)  suggests  that  by  its  very  nature  play  is  intrinsically  motivating;  learners’ 
motivational  orientation  may  have  therefore  been  enhanced  simply  because  they  felt  they  were  able  to  spend  three 
weeks  of  science  class  playing. 

Novelty 

The  second  theme  that  emerged  from  the  data  has  to  do  with  the  impact  that  the  novelty  of  this  environment 
may  have  on  students'  motivational  orientation.  In  addition  to  pointing  out  how  Alien  Rescue  differed  from  their 
regular  class  activities  within  the  other  themes,  students’  comments  showed  that  they  enjoyed  the  change  in  routine 
that  their  work  on  this  program  represented.  Comments  such  as  "I’ve  never  done  something  like  that,  " "I  liked 
getting  away  from  class  for  a while,  " "It’s  better  than  working  in  a book,"  and  "We  don't  usually  get  to  use 
computers,"  show  that,  for  students,  one  positive  aspect  of  this  program  was  simply  that  it  was  different  from  regular 
classes.  One  student  explained  that  typical  class  activities  are  of  limited  duration,  and  that  he  like  the  prolonged 
investigation  offered  by  the  software:  "It  was  for  an  extended  amount  of  time,  not  just  one  day,  so  when  you  go  in 
the  next  day  and  you  can  say  'Oh,  where  was  I?  Oh,  yeah,  I’ve  got  to  get  information  from  the  probe.'  I liked  that." 
Some  comments  suggest  that  students  found  their  work  in  this  environment  less  intimidating  than  in  their  regular 
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classes:  "It  was  easier,  it  was  fun,  cause  you  usually  don't  get  to  talk  in  class.  Be  quiet  and  if  you  don't  be  quiet  you 
get  a detention.  But  we  were  allowed  to  talk."  And  "We  get  an  automatic  100  everyday  instead  of  getting  like  70s 
and  stuff."  The  lack  of  extrinsic  motivators,  such  as  punishment  and  grades,  may  have  freed  students  to  adopt  a 
more  intrinsic  orientation  than  they  do  in  their  regular  classes. 

Discussion 

The  findings  of  this  study  strengthen  the  growing  literature  on  the  effectiveness  of  problem-based  learning 
in  two  ways.  First,  it  shows  that  PBL  can  encourage  intrinsically  motivated  behavior.  Second,  these  results  were 
achieved  with  regular  education  sixth  graders,  suggesting  that  PBL  should  be  used  with  a wider  audience  than  has 
been  the  case  in  the  past. 

These  findings  also  lend  support  for  the  notion  of  the  situated  nature  of  motivation.  Students'  scores  on  the 
post  treatment  administration  of  the  Scale  of  Intrinsic  versus  Extrinsic  Orientation  in  the  Classroom  were  correlated 
with  their  pre  treatment  scores,  indicating  that  there  are  enduring  motivational  characteristics  of  learners  that  affect 
how  they  react  to  different  learning  environments.  At  the  same  time,  the  significant  difference  between  the  scores 
on  the  two  administrations  of  this  scale  indicate  that  some  environments  are  inherently  more  motivating  than  others. 
Taken  together,  these  findings  support  the  argument  that  motivation  is  a product  of  a person  by  context  interaction. 

Why  do  students  report  a more  intrinsic  orientation  for  an  environment  like  the  one  posed  by  Alien  Rescue 
than  for  their  regular  class  environments?  The  qualitative  data  suggest  that  challenge,  control,  and  collaboration,  as 
Paris  and  Turner  suggest,  impact  motivation  positively.  A PBL  approach  may  also  foster  the  perception  in  students 
that  they  are  learning  valuable  problem-solving  skills  and  working  as  scientists;  seeing  the  value  in  their  activities 
may  enhance  motivation,  as  widely  suggested  in  the  literature  on  motivation.  However,  other  factors  may  partially 
account  for  the  high  level  of  intrinsic  motivation  students  reported.  Students  used  the  computer  lab  everyday  for 
three  weeks,  which  they  had  never  before  done  as  part  of  a class.  Unfettered  by  concerns  over  grades  or  detentions, 
students  may  have  felt  more  willing  to  take  risks  and  enjoy  the  challenging  nature  of  the  tasks  involved  in 
developing  a solution  plan.  The  interactive  nature  and  rich  media  of  the  computer  program  may  have  promoted  a 
sense  of  play  and  curiosity.  These  factors,  as  well  as  the  sheer  novelty  of  the  approach  and  the  software,  may  have 
encouraged  students  to  react  with  greater  intrinsic  motivation  than  they  do  in  their  regular  science  classes.  Certainly 
they  suggest  a need  for  a closer  investigation  of  the  attributes  of  this  environment  as  well  as  the  environments 
created  by  other  PBL  programs  to  determine  which  contextual  factors  unite  to  support  an  intrinsic  orientation. 
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Abstract 

Corporate  and  educational  settings  increasingly  require  decision-making,  problem-solving  and  other 
complex  cognitive  skills  to  handle  ill-structured , or  heuristic,  tasks , but  the  growing  need  for  heuristic  task  expertise 
has  outpaced  the  refinement  of  task  analysis  methods  for  heuristic  expertise.  The  Heuristic  Task  Analysis  Method 
was  applied  to  three  settings  to  generate  improvements  and  more  detailed  guidance,  and  to  identify  variations  in  the 
method  for  different  situations.  The  three  settings  were  group  counseling,  tutoring  on  writing  skills,  and  selecting 
artwork  for  a product  line.  The  formative  research  methodology  was  used  to  test  the  method  and  generate 
improvements.  The  three  studies  produced  some  common  and  some  unique  findings  and  recommendations.  A 
tentative  revision  to  the  HTA  method  is  proposed. 


Introduction 

As  our  society  in  general  and  the  workplace  in  particular  become  more  complex,  we  are.  finding  that  a 
greater  number  of  the  activities  that  people  undertake  are  relatively  more  heuristic  in  nature  than  ever  before. 
Whether  in  K-12  education,  higher  education,  corporate  training,  or  any  other  context,  to  help  people  learn  the 
heuristic  elements  of  an  expert's  know-how,  we  must  be  able  to  identify  those  heuristics. 

In  analyzing  heuristics,  we  find  it  helpful  to  think  in  terms  of  two  major  kinds  of  expertise — domain  and 
task  expertise.  Task  expertise  relates  to  the  learner  becoming  an  expert  in  a specific  task,  such  as  managing  a 
project,  selling  a product,  or  writing  an  annual  plan.  Domain  expertise  relates  to  the  learner  becoming  an  expert  in  a 
body  of  subject  matter  not  tied  to  any  specific  task,  such  as  economics,  electronics,  or  physics  (but  often  relevant  to 
many  tasks).  (Reigeluth,  1999,  p.  435). 

Both  procedural  and  declarative  knowledge  are  important  elements  of  both  kinds  of  expertise.  In  this  research,  we 
focus  on  task  expertise.  For  task  expertise,  we  find  it  helpful  to  think  in  terms  of  two  major  kinds  of  tasks: 
procedural  and  heuristic.  Procedural  tasks  are  "tasks  for  which  experts  use  a set  of  steps,  mental  and/or  physical,  to 
decide  what  to  do  when,  such  as  a high  school  course  on  mathematics  or  a corporate  training  program  on  installing  a 
piece  of  equipment  for  a customer.  Heuristic  tasks  are  "tasks  for  which  experts  use  causal  models — interrelated  sets 
of  principles  and/or  guidelines — to  decide  what  to  do  when,  such  as  a high  school  course  on  thinking  skills  or  a 
corporate  training  program  on  management  skills."  (Reigeluth,  1 999,  p.  435). 

The  distinction  between  procedural  and  heuristic  tasks  is  similar  to  the  distinction  between  well  structured 
and  ill  structured  domains  (Fredericksen,  1984;  Resnick,  1988;  Simon,  1973).  In  reality  most  tasks  are  neither 
purely  procedural  nor  purely  heuristic,  but  some  combination  of  the  two.  We  have  relatively  powerful 
methodologies  for  analyzing  the  expertise  that  underlies  procedural  tasks  (i.e.  the  mental  and  physical  steps  upon 
which  an  expert  relies).  But  we  do  not  have  good  methodologies  for  analyzing  the  expertise  that  underlies  heuristic 
tasks.  This  situation  is  exacerbated  by  the  reality  that  heuristic  knowledge  is  frequently  tacit  rather  than  explicit — 
that  is,  experts  are  often  unaware  of  the  heuristics  that  guide  their  performance.  Therefore,  there  is  a strong  need  to 
develop  task  analysis  methodologies  for  identifying  the  knowledge  that  underlies  heuristic  tasks.  The  full  paper 
reviews  literature  related  to  heuristic  task  analysis.  Then  it  describes  three  research  studies  that  have  been 
conducted  to  improve  one  of  those  methodologies.  This  paper  is  a summary  of  the  full  paper. 

The  Heuristic  Task  Analysis  Method  of  Elaboration  Theory 

The  Elaboration  Theory  (Reigeluth,  1999;  in  press)  offers  the  Heuristic  Task  Analysis  (HTA)  method  as 
part  of  its  Simplifying  Conditions  Method  (SCM)  for  task  analysis.  This  more  general  method  of  task  analysis  is 
conducted  by  asking  the  question,  "What  is  the  simplest  version  of  the  task  that  an  expert  has  ever  performed?"  and 
"What  is  the  next  simplest  version?"  and  so  forth.  As  each  version  is  identified,  its  place  in  the  learning  sequence  is 
simultaneously  determined.  Therefore,  the  SCM  task  analysis  method  is  an  integral  part  of  the  method  for  designing 
an  instructional  sequence.  Furthermore,  since  most  tasks  have  a combination  of  procedural  and  heuristic  elements, 
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the  procedural  and  heuristic  task  analysis  methods  are  integrated  into  a single  process.  For  these  reasons  and 
because  this  method  offers  a fair  amount  of  guidance  for  the  task  analysis  process,  we  chose  this  method  for  our 
research.  Hence,  it  is  described  in  greater  detail  next. 

The  SCM’s  Heuristic  Task  Analysis  Method 

The  following  are  some  details  on  the  SCM's  heuristic  task  analysis  method.  They  are  an  elaboration  of  the 
process  described  by  Reigeluth  (1999). 

Phase  I.  Prepare  for  Analysis  and  Design 

1.  Prepare.  Lay  the  groundwork  for  your  analysis  and  design. 

1 . 1 Establish  rapport  with  a task  expert. 

1 .2  Explain  the  analysis  process  you  will  be  using. 

1 .3  Ask  the  task  expert  about  the  nature  of  the  task  in  general. 

1 .4  Identify  the  characteristics  of  the  learners  in  general. 

1 .5  Identify  the  delivery  constraints  of  the  task  in  general. 

Phase  II.  Identify  the  First  Learning  Episode 

2.  Identify  the  simplest  version.  Help  the  task  expert  to  identify  the  simplest  version  of  the  task  that  is  fairly 
representative  of  the  task  as  a whole,  and  to  describe  the  conditions  that  distinguish  that  version  from  all  other 
versions. 

1.  You  may  want  to  use  some  other  criteria  in  addition  to  simple  and  representative,  such  as  common  and 
safe 

2.  Ask  the  task  expert  to  recall  the  simplest  case  she  or  he  has  ever  seen.  The  simplest  version  will  be  a class 
of  similar  cases.  Then  check  to  see  how  representative  it  is  of  the  task  as  a whole. 

3.  It  may  be  helpful  to  start  by  identifying  some  of  the  major  versions  of  the  task  and  the  conditions  that 
distinguish  when  one  version  is  appropriate  versus  another. 

4.  Thinking  of  different  conditions  helps  to  identify  versions,  and  thinking  of  different  versions  helps  to 
identify  conditions.  Hence,  it  is  wise  to  do  both  simultaneously  (or  alternately). 

5.  There  is  no  single  right  version  to  choose  as  the  "simplest."  It  is  usually  a matter  of  trade-offs.  The  very 
simplest  version  of  the  task  is  usually  not  very  representative  of  the  task  as  a whole.  The  more 
representative  the  simple  version  can  be,  the  better,  for  it  provides  a more  useful  schema  to  which  learners 
can  relate  subsequent  versions. 

6.  It  may  be  wise  to  go  through  this  process  with  several  task  experts  before  going  on  to  Step  3.  You  may 
find  it  necessary  to  take  steps  to  resolve  differences  of  opinion  about  which  is  the  best  “simplest  version”  to 
use. 

3.  Analyze  the  organizing  content.  Analyze  the  organizing  content  for  this  version  of  the  task. 

3.1  Ask  the  task  expert  to  think  of  one  specific  performance  of  the  task  to  analyze,  or  to  videotape  a 
performance  for  you  to  review  with  the  expert  during  the  analysis. 

3.2  Use  a top-down  approach  to  analyzing  the  content  (the  knowledge  upon  which  the  expert’s  performance  is 
based).  In  other  words,  start  by  identifying  the  general  categories  of  knowledge  that  an  expert  uses,  then 
proceed  to  analyze  each. 

7.  Ask  the  task  expert: 

- to  describe  each  decision  that  the  task  expert  made, 

- to  identify  the  kinds  of  knowledge  that  the  task  expert  drew  upon  to  make  the  decision,  and 

- to  describe  the  specific  knowledge  within  each  kind  of  knowledge  that  the  task  expert  used. 

8.  The  kinds  of  knowledge  are  likely  to  include: 

- steps  (procedural  knowledge), 

- guidelines  or  rules  of  thumb  (heuristic  knowledge), 

- explanatory  models  (which  explain  why  the  guidelines  work), 

- descriptive  models  (which  describe  the  phenomena  with  which  the  task  expert  interacts),  and 

- metacognitive/decision  rules  (which  the  task  expert  uses  to  decide  which  steps,  guidelines,  and 
descriptive  knowledge,  to  use  when). 

9.  It  is  generally  helpful  to  start  by  asking  the  task  expert  if  there  are  any  steps  or  phases  of  activities  that 
are  always  performed  for  this  version  of  the  task.  If  so,  perform  a procedural  task  analysis  to  identify 
the  sequence  of  steps  and  to  see  if  any  of  those  steps  can  be  broken  down  into  substeps,  but  those 
substeps  must  be  ones  that  an  expert  thinks  of  and  uses  routinely  in  performing  that  version  of  the  task. 

10.  For  guidelines,  use  the  following  process: 

1 . Identify  the  goals  for  this  specific  performance  of  the  task  under  its  conditions. 
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2.  Identify  all  the  important  considerations  for  attaining  each  goal.  Considerations  are  the  major 
categories  of  causal  factors  that  influence  performance  of  the  task.  If  there  are  a lot  of  causal  factors 
for  a consideration,  it  is  useful  to  identify  subconsiderations  for  it. 

3.  Identify  all  the  important  causal  factors  for  each  consideration  (or  subconsideration). 

4.  Analyze  each  causal  factor  to  identify  all  guidelines  (prescriptive  principles  or  “rules  of  thumb”)  that 
an  expert  uses  to  account  for  this  consideration. 

1 l.For  explanatory  models,  use  the  following  process: 

12.  For  each  guideline,  ask  the  task  expert  for  the  reasons  why  s/he  believes  it  works. 

13.  For  interrelated  guidelines,  you  are  likely  to  identify  a set  of  related  reasons  that  constitute  a causal 
model  or  models.  Be  sure  to  look  for  multiple  causes  for  each  effect  and  multiple  effects  for  each  cause. 
Also  look  for  chains  of  causes  and  effects,  and  explore  probabilities  for  each  causal  factor  to  have  each 
effect. 

14.  For  descriptive  models,  use  the  following  process: 

1.  Ask  the  task  expert  what  phenomena  influenced  this  particular  performance  of  the  task.  Try  to 
identify  all  causal  relationships  that  characterized  those  phenomena.  Be  sure  to  look  for  multiple 
causes  for  each  effect  and  multiple  effects  for  each  cause.  Also  look  for  chains  of  causes  and  effects, 
and  explore  probabilities  for  each  causal  factor  to  have  each  effect. 

15.  For  metacognitive/decision  rules,  use  the  following  process: 

1.  Find  out  what  rules  the  task  expert  used  to  decide  when  to  use  which  steps,  guidelines,  and 
descriptive  models  during  the  specific  performance  of  the  task  being  analyzed. 

16.  It  is  wise  to  query  the  task  expert  about  any  of  these  kinds  of  knowledge  that  are  not  initially  described 
to  you  for  each  decision  the  task  expert  made  in  this  specific  performance  of  the  task. 

3.3  Ask  the  task  expert  to  think  of  similar  performances  of  the  task  that  constitute  a single  version  of  the  task. 
Use  each  such  performance  to  broaden  the  steps,  guidelines,  explanatory  models,  descriptive  models,  and 
metacognitive/decision  rules  so  that  they  represent  the  knowledge  bases  the  task  expert  uses  to  deal  with  all 
performances  for  that  version  of  the  task. 

3.4  If  time  and  resources  permit,  find  a second  task  expert  with  whom  to  repeat  this  entire  process  (Steps  1- 
3.3)  to  identify  any  alternative  views  of  the  task  and  the  knowledge  that  underlies  its  performance.  It  may 
even  be  wise  to  repeat  this  process  with  several  more  task  experts.  And  you  may  want  to  go  back  and  ask 
each  task  expert  what  s/he  thinks  about  the  perspectives  of  the  other  task  experts,  in  an  effort  to  reconcile 
conflicts  and  select  among  alternative  ways  of  thinking  about  and  performing  the  task. 

The  HTA  method  has  not  been  rigorously  tested  and  therefore  is  in  need  of  further  research.  However,  the 
most  important  research  issue  is  not  the  validity  of  the  HTA  method,  for,  like  most  methods,  it  is  likely  to  work 
some  times  but  not  always,  and  to  varying  degrees.  Rather,  given  the  immaturity  of  our  knowledge  about  how  to 
analyze  heuristic  tasks,  what  is  needed  most  at  this  point  is  developmental  research — research  that  is  intended  to 
further  develop  and  improve  the  method.  Therefore,  our  research  question  is,  "In  what  ways  can  the  HTA  method 
possibly  be  improved?"  To  answer  this  question,  it  is  also  necessary  to  find  out  what  parts  of  the  method  are 
working  well  and  what  parts  are  not  working  so  well.  Furthermore,  to  improve  the  HTA  method,  it  will  likely  be 
important  (1)  to  change  parts  of  the  method,  (2)  to  provide  more  detailed  guidance  about  how  to  accomplish 
particular  parts  of  the  method,  and  (3)  to  identify  variations  in  the  method  for  different  situations,  such  as  different 
kinds  of  tasks  or  even  different  kinds  of  task  experts. 

To  answer  these  research  questions,  we  conducted  a series  of  three  developmental  research  studies.  These 
are  described  next,  followed  by  some  general  conclusions. 

Study  1:  The  HTA  Method  Applied  to  Group  Counseling 

The  purpose  of  this  study  was  to  improve  the  HTA  method  and  guidance  for  use  of  that  method.  Thus,  the 
formative  research  methodology  was  adopted  with  emphasis  on  exploring  how  the  HTA  method  can  be  improved 
when  applied  to  group  counseling. 

Formative  research  is  a kind  of  developmental  research  or  action  research  that  is  intended  to  improve 
design  theory  (Reigeluth  & Frick,  1999).  In  contrast  to  research  on  descriptive  theory,  which  emphasizes  validity  or 
how  well  the  description  matches  the  reality  of  "what  is,"  research  on  design  theory  is  more  concerned  with 
preferability,  the  extent  to  which  one  method  is  better  than  other  methods  for  achieving  certain  goals  under  certain 
circumstances.  By  creating  or  identifying  an  instance  of  a design  theory  and  collecting  formative  data  to  improve 
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that  instance,  one  may  develop  better  understanding  of  how  the  theory  works  in  the  field  and  thus  be  able  to  propose 
improvements  for  the  theory,  which  of  course  would  need  to  undergo  further  testing. 

Task.  Group  counseling  was  selected  as  the  task  to  which  to  apply  the  HTA  method.  Group  counseling  is 
a combination  task  with  both  procedural  and  heuristic  elements.  It  is  procedural  in  that  the  activities  of  the  group 
leader  are  largely  determined  by  the  stages  that  a group  goes  through  (i.e.,  forming,  norming,  storming,  and 
performing),  and  the  leader  cannot  help  the  group  to  progress  to  the  next  stage  without  performing  certain  tasks 
(steps)  at  each  stage.  However,  at  a deeper  level  of  analysis,  the  knowledge  required  for  the  leader  to  decide  when 
and  how  to  intervene  is  not  a set  of  steps  but  a set  of  guidelines  and  principles,  which  are  heuristic  knowledge. 

Participants.  This  study  involved  three  participants  as  task  experts  in  group  counseling.  Expert  #1  was  a 
professor  in  the  Counseling  Department  in  the  Indiana  University  School  of  Education  and  was  the  most 
experienced  of  the  three  task  experts.  The  other  two  were  doctoral  students  in  the  same  department.  They  were  all 
experienced  in  conducting  "personal  growth"  group  counseling,  and  their  expertise  ranged  from  three  to  more  than 
20  years. 

Data  Collection  Methods 

Interview.  Semi-structured  in-person  interviews  were  used  as  the  primary  data  collection  method.  The 
second  author  conducted  six  interviews  between  September  and  November,  1999.  Each  interview  took  30-90 
minutes  and  was  audio  taped  for  analysis.  The  purpose  of  the  interviews  was  to  find  ways  to  improve  the  HTA 
method  for  eliciting,  analyzing,  and  representing  the  expert's  heuristic  knowledge  for  performing  the  task  of 
"personal  growth"  group  counseling.  The  investigator  played  two  roles,  one  as  a task  analyst  proficient  in  the  HTA 
method  and  the  other  as  researcher  searching  for  ways  to  improve  the  HTA  method.  As  task  analyst,  the 
investigator  developed  a set  of  interview  questions  (see  Appendix  A)  for  the  interview  based  on  the  HTA  method, 
but  as  researcher  the  investigator  was  not  restricted  to  the  predefined  questions.  Depending  on  the  expert’s  response, 
the  researcher  revised  the  HTA  method  for  the  next  interview.  Thus,  the  overall  interview  process  was  flexible  and 
reflective  in  nature. 

Videotapes.  Because  of  the  confidential  nature  of  group  counseling,  direct  observation  or  videotaping  of 
an  expert’s  task  performance  (as  called  for  by  the  HTA  method)  was  not  allowed.  Instead,  the  researcher  (as 
analyst)  used  a series  of  instructional  videos  that  simulated  group  counseling  sessions  for  beginning  group  leaders, 
to  provide  the  analysts  with  a concrete  case. 

Data  Analysis  and  Interpretation  Methods 

The  HTA  method  is  an  iterative  process:  finishing  the  first  round  of  HTA  is  not  the  end  of  the  study  but  the 
beginning  of  the  second  round  of  HTA;  and  the  end  of  the  second  round  is,  again,  the  beginning  of  the  third  round; 
and  so  on.  The  investigator  went  through  two  rounds  of  HTA  in  this  study. 

Triangulation.  To  enhance  the  thoroughness  of  the  data,  this  study  involved  three  experts  as  data  sources. 
Each  of  them  played  somewhat  different  roles  during  the  interviews.  During  each  round  of  data  collection,  expert 
#2  provided  the  initial  structure  of  the  task  setting  and  knowledge  base.  Then  experts  #1  and  #3  reviewed  the 
knowledge  elements,  verified  them,  and  provided  additional  information.  There  were  a few  times  when  the  three 
experts  did  not  agree  with  one  another.  In  such  cases,  expert  #l's  judgment  was  accepted,  as  he  was  the  most 
experienced  group  counselor. 

Member  checks.  After  each  interview  with  an  expert,  the  researcher  transcribed  the  interview  and  took  the 
summary  and  interpretations  to  the  next  interview  for  review.  Through  this  process,  the  experts  corrected  errors  or 
misconceptions  by  the  researcher,  and  the  researcher  asked  additional  questions  to  clarify  the  information. 

Consultation.  During  the  data  collection  and  analysis  process,  the  researcher  regularly  met  with  the  other 
three  researchers  in  this  study  and  consulted  them  in  designing  the  interview  protocol  and  analyzing  the  data. 

Results  and  Discussion 

The  first  round  of  data  collection  involved  initial  interviews  with  three  experts.  Instead  of  finishing  with 
one  expert  and  then  starting  with  another,  the  researcher  worked  with  the  three  experts  simultaneously  (but 
separately)  due  to  their  time  schedules.  This  approach  involved  some  tradeoffs.  It  worked  well  in  the  sense  that  the 
researcher  could  get  the  three  experts  to  reach  consensus  on  the  simplest  version  of  the  task  early  in  the  HTA 
process.  However,  communicating  with  all  three  experts  simultaneously  was  not  easy  for  the  researcher,  and  the 
researcher  had  to  spend  most  of  the  time  during  the  interviews  explaining  to  each  expert  the  previous  interviews 
with  other  experts.  Even  though  the  task  was  a common  one,  the  experts  still  had  difficulty  explaining  the  detailed 
decision-making  process  when  the  task  had  originally  been  defined  by  another  expert.  Facing  this  problem,  the 
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researcher  decided  to  use  an  existing  instructional  video  series  (with  which  all  three  experts  were  familiar)  as  a 
frame  of  reference,  instead  of  trying  to  build  a new  scenario  based  on  each  expert's  experience. 

The  second  problem  was  that  the  researcher  lacked  expertise  in  the  task  of  group  counseling.  The 
researcher  found  that,  to  be  able  to  push  the  expert  to  further  elaborate  his/her  automatized  (and  hence  subconscious) 
task  expertise,  the  analyst  needed  to  speak  the  same  language  as  the  expert  and  be  able  to  prompt  when  the  expert 
had  difficulty  in  finding  the  right  words.  Without  a certain  level  of  expertise  in  the  task,  the  researcher  as  task 
analyst  had  difficulty  doing  those  jobs  smoothly. 

The  third  problem  was  related  to  the  difficulty  of  categorizing  the  types  of  knowledge  underlying  each 
decision  made  by  the  expert  during  the  task  analysis.  The  purpose  of  identifying  the  five  types  of  knowledge 
identified  by  the  HTA  was  to  make  sure  that  the  expert  did  not  overlook  one  of  the  important  types  of  knowledge, 
but  the  benefits  of  distinguishing  among  the  types  seemed  to  not  be  worth  the  extra  time  required  in  this  case. 

The  second  round  of  HTA  incorporated  some  new  methods  to  deal  with  the  problems  found  in  the  first 
round.  First,  the  researcher  summarized  key  incidents  from  the  video  series  on  index  cards  and  used  them  as  a 
reference  during  the  interviews  with  the  experts.  This  was  very  helpful  in  three  ways:  (1)  it  helped  the  experts  to 
recall  details  about  the  task  performance  process,  (2)  it  helped  both  analyst  and  experts  to  see  the  flow  of  the  task 
performance  process  and  get  back  on  track  when  the  experts  got  off-task,  and  (3)  it  saved  a lot  of  time  in  revisiting 
previous  points.  One  expert  commented  that  the  index  cards  forced  him  to  be  more  precise  during  the  review  and 
revision  process.  Second,  the  researcher  as  analyst  used  a bottom-up  approach  (identifying  knowledge  first,  then 
categorizing  it  as  to  type)  rather  than  the  top-down  approach  (identifying  knowledge  within  each  type)  suggested  by 
the  HTA  method. 

Based  on  the  findings  of  this  study,  the  following  changes  are  proposed  as  possible  improvements  and 
described  in  detail  below:  (1)  incorporate  various  interview  and  observation  techniques  into  the  HTA  process,  (2) 
provide  different  guidelines  for  analysts  with  different  levels  of  task  expertise,  (3)  provide  different  guidelines  for 
working  with  task  experts  with  different  levels  of  expertise,  and  (4)  provide  reference  material  during  the  interview 
with  task  experts. 

Study  2 - The  HTA  Method  Applied  to  Tutoring  on  Writing  Skills 

As  in  Study  1,  the  purpose  of  this  study  was  to  improve  Reigeluth’s  HTA  process  by  using  the  formative 
research  methodology.  This  study  followed  the  steps  suggested  by  Reigeluth  and  Frick  outlined  in  study  1. 

Task.  The  heuristic  task  chosen  for  this  study  was  tutoring  university  undergraduate  students  who  needed 
extra  assistance  with  their  writing  skills.  Specifically,  the  task  concerned  the  decision-making  process  in  which  an 
expert  writing  tutor  engages  to  determine  the  direction  and  focus  of  the  tutoring  session.  By  its  very  nature,  a 
tutoring  session  requires  a lot  of  heuristic  expertise,  because  it  is  determined  more  by  events  that  occur  during  the 
tutoring  event  than  by  any  predetermined  procedural  steps.  What  occurs  during  the  tutoring  session  depends  on  both 
the  writing  situation  and  the  tutee.  The  writing  situation  includes  why  the  tutee  is  being  tutored,  the  relationship 
between  the  tutee  and  the  teacher,  the  interest  level  of  the  topic  being  written  about,  and  the  number  of  drafts  already 
written.  The  tutee  includes  any  previous  experiences,  both  positive  and  negative,  that  the  tutee  brings  to  the  tutoring 
session. 

Participants.  Two  experts  were  chosen  based  on  their  level  of  expertise  and  the  approval  of  their 
supervisors.  Both  experts  had  extensive  experience  tutoring  all  levels  of  writing  students,  and  both  were  highly 
recommended  by  their  writing  center  supervisors.  A third  tutor  was  also  recommended  and  interviewed  as  a 
potential  participant  in  this  study.  However,  the  recommendation  came  without  the  experience  and  evaluation 
credentials  listed  above,  so  he  was  not  included  in  this  study. 

Data  Collection  Methods 

Interview:  As  in  study  1,  the  main  data  gathering  method  was  the  personal  interview,  and  the  researcher 
both  elicited  heuristic  knowledge  (analyst  role)  and  conducted  formative  research  (researcher  role).  Two  interviews 
were  conducted  within  one  week  of  the  tutoring  session  that  was  being  analyzed.  Both  of  the  interviews  were 
conducted  within  one  week  of  the  tutoring  session  that  was  being  analyzed.  Because  of  the  tutors’  lack  of  time  to 
spend  on  this  research,  each  interview  was  limited  to  approximately  60  minutes.  Prior  to  the  interviews,  each  of  the 
tutors  was  sent  emails  describing  terms  used,  definitions,  an  outline  of  the  interview  questions,  and  a brief 
explanation,  written  in  their  terms,  of  the  purpose,  expected  results,  and  use  of  this  research. 

Before  the  actual  interview,  the  researcher  reminded  each  tutor  of  what  was  sent  to  them  earlier  and  asked  if  any 
terms  or  points  needed  to  be  clarified.  At  this  time,  the  researcher  also  pointed  out  to  each  expert  writing  tutor  that 
a)  it  was  unclear  whether  the  task  about  to  be  analyzed  was  actually  based  on  heuristic  knowledge  and  b)  it  was 
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unclear  whether  the  questions  would  be  able  to  access  that  knowledge.  This  was  done  to  reduce  any  anxiety  that  the 
writing  tutors  might  experience  if  they  could  not  produce  information  that  the  researcher  desired. 

Based  on  the  first  interview,  the  HTA  methodology  was  altered  slightly  for  the  second  interview  so  as  to 
assist  the  tutor  to  better  recall  the  tutoring  situation.  The  analyst  had  the  tutor  respond  to  specific  questions  about  the 
actual  tutoring  experience,  the  tutee’s  characteristics,  and  the  tutee’s  essay  prior  to  having  the  tutor  recall  and  reflect 
on  her  decision-making  processes.  He  then  had  the  tutor  identify  the  decision  areas  to  focus  on  during  the  tutoring 
session.  Afterwards,  he  had  the  tutor  choose  the  concern  that  was  most  available  to  her.  This,  then,  became  the 
subject  of  the  heuristic  task  analysis. 

Data  Analysis  and  Interpretation  Methods 

There  were  no  follow-up  interviews  or  member  checks  as  in  Study  1 to  determine  the  validity  of  the  tutors’ 
responses  due  to  the  writing  tutors’  lack  of  time  to  spend  on  these  tasks.  However,  because  of  the  researcher’s 
expertise  in  this  area,  he  concluded  that  the  data  collected  was  not  spurious.  At  the  conclusion  of  each  interview,  the 
experts  were  asked  to  review  and  modify  what  was  recorded  during  the  interview.  The  researcher  asked  each  expert 
for  ways  to  improve  the  interview  process  and  to  comment  on  its  effectiveness  in  eliciting  the  knowledge  underlying 
their  decision-making  thought  process.  Both  offered  suggestions  about  ways  to  help  them  recall  the  previous 
tutoring  situation  and  about  the  limitation  of  focusing  on  only  one  aspect  of  the  tutoring  process.  The  second  expert 
confirmed  what  the  first  had  concluded.  After  each  interview,  tentative  changes  were  made  to  the  HTA  process. 

Results  and  Discussion 

In  the  interview  after  the  first  application  of  the  HTA  method,  the  first  tutor  mentioned  how  the  process 
helped  him  think  about  his  own  tutoring  strategies.  He  also  mentioned  that  having  to  recall  from  memory  a tutoring 
session  that  was  done  even  within  the  last  seven  days  was  not  easy.  The  tutor  suggested  the  following  refinements 
to  the  HTA  method.  1)  The  top-down  process  seemed  effective.  2)  Because  the  tutor  experienced  some  difficulty 
recalling  the  specific  tutoring  session,  the  analyst  (researcher)  asked  some  specific  questions  about  the  tutee,  the 
paper,  and  the  tutee’s  reactions  to  the  tutor’s  suggestions.  Both  tutors  said  this  helped  them  get  into  the  flow  of  the 
previous  tutoring  session,  and  the  researcher  observed  a marked  increase  in  awareness  and  confidence  after  assisting 
the  tutor’s  recall.  3)  5x8  cards  were  effective  in  that  the  tutor  referred  back  to  them  to  align  his  insights  into  the 
tutoring  process  with  previous  statements.  4)  The  tutor,  when  identifying  the  guidelines,  focused  more  on  how  to 
hold  an  effective  tutoring  session  than  on  what  influenced  his  decision  to  focus  on  a specific  tutoring  objective.  In 
addition,  5)  the  researcher  suspected  that  the  results  of  the  HTA  might  have  been  richer  if  the  task  expert  (tutor)  had 
been  given  more  control  over  the  decision  point  selection  process. 

In  summary,  an  important  concern  involves  the  task  expert’s  tendency,  when  explicating  the  guidelines,  to 
focus  on  the  goals  and  not  on  the  decision  points  for  attaining  the  goals.  During  both  instantiations,  the  writing 
tutors  gave  the  guidelines  they  used  for  deciding  on  the  goals  of  the  tutoring  situation  rather  than  giving  guidelines 
for  deciding  how  to  attain  a goal  during  the  tutoring  session.  When  this  occurred,  the  researcher  gently  prodded  the 
experts  to  focus  on  the  decision  points  rather  than  the  goals.  However,  when  the  experts  could  not  provide  that 
information,  the  researcher  decided  to  review  previous  sections  and  then  ask  that  question  again.  After  the  experts 
referred  to  the  goals  again,  the  researcher  decided  not  to  push  them  any  further,  seeing  that  they  both  were  unable  to 
provide  that  information.  Another  concern  involves  the  first  expert’s  difficulty  in  recalling  the  tutoring  session 
despite  the  fact  that  the  session  occurred  only  one  week  prior  to  the  interview.  Measures  taken  to  assist  the  second 
expert’s  ability  to  recall  the  tutoring  session  showed  a dramatic  improvement. 

Study  3:  Selecting  Artwork  for  a Commercial  Product  Line 

The  third  study  tested  the  HTA  method  in  a corporate  setting.  As  in  the  previous  two  studies,  formative 
research  was  the  methodology,  using  a designed  case  to  generate  possible  improvements  in  the  HTA  method. 
Corporate  executives  want  a “big  bang  for  their  buck,”  and  analysis  is  often  looked  upon  as  a time-consuming 
activity  with  questionable  impact.  The  aim  of  this  study  was  to  develop  a rapid,  high-impact  version  of  the  HTA 
method  for  corporate  settings.  Thus,  the  study  was  designed  to  provide  insight  into  the  following  research  questions: 
(1)  How  can  the  speed  and  effectiveness  of  the  HTA  method  be  improved  for  eliciting,  analyzing,  and  representing 
heuristic  knowledge  from  experts  in  corporate  settings?  (2)  What  guidance  could  be  added  to  the  method  to  assist 
analysts  in  corporate  settings?  The  time  constraints  for  this  study  dictated  that  the  research  be  limited  to  a single 
interview  cycle  with  one  task  expert,  lasting  no  more  than  a total  of  three  to  four  hours. 

Task.  The  heuristic  task  chosen  for  this  study  was  deciding  whether  a submission  of  artwork  was  suitable 
for  one  of  the  company’s  product  lines.  This  was  a judgmental  decision-making  task  requiring  a fair  amount  of 
experience  and  know-how.  The  task  expert  verified  that  the  task  was  important  to  the  company,  that  she  was 


considered  to  be  an  expert  at  the  task,  and  that  it  was  not  easy  to  articulate  the  expertise  required  to  perform  the  task. 
The  heuristic  nature  of  this  task  was  verified  by  an  expert  in  the  HTA  method. 

Participant.  The  task  expert  was  recruited  by  calling  a local  business  that  had  collaborated  with  the 
Instructional  Systems  Technology  Department  at  Indiana  University  in  the  past.  A manager  in  the  design  department 
enthusiastically  agreed  to  participate  in  the  study.  The  expert  and  the  analyst  discussed  possible  complex  decision- 
making tasks  over  the  telephone  and  came  to  an  agreement  on  an  appropriate  task. 

Data  Collection,  Analysis,  and  Interpretation  Methods 

Interview.  The  researcher/analyst  conducted  two  audio  taped,  one-and-a-half-hour  interviews  with  the 
participant  (task  expert)  in  a conference  room  at  the  expert’s  place  of  business.  The  analyst/researcher  referred  to  an 
interview  sheet  (described  below)  to  ensure  that  he  was  adhering  to  the  guidelines  of  the  HTA  method,  although  he 
also  allowed  the  interview  to  be  somewhat  unstructured  as  seemed  appropriate  to  gather  the  heuristic  knowledge  and 
data  for  improving  the  HTA  process.  After  one-and-a-half  hours  he  reached  a saturation  point  in  terms  of  gathering 
the  essence  of  the  task  and  the  key  heuristics  and  concluded  he  could  not  effectively  continue  the  analysis  without 
first  going  back  to  his  office  and  organizing  the  information  collected.  The  expert  agreed  to  continue  the  interview 
the  following  week.  The  analyst/researcher  logged  “significant  chunks”  of  the  audio  tape  on  3”  x 5”  cards.  His 
criterion  for  “significant  chunks”  was  any  piece  of  knowledge  that  fit  into  one  or  more  of  the  types  of  knowledge 
listed  in  the  HTA  method.  He  examined  these  knowledge  elements  to  determine  what  missing  ones  he  needed  to  ask 
about  in  the  follow-up  interview.  Then  in  the  role  of  researcher,  he  re-examined  the  interview  results  for  deviations 
from  the  HTA  method  to  see  where  the  method  was  effective  in  eliciting  heuristic  expertise  and  where  deviations 
were  helpful.  He  discussed  his  preliminary  findings  with  Reigeluth  and  worked  with  him  to  plan  the  second 
interview. 

Results  and  Discussion 

Speed  of  the  HTA  method.  The  analyst/researcher  found  a number  of  areas  in  which  the  speed  of  the 
HTA  method  might  be  enhanced.  Two  are  discussed  in  this  section.  The  others  are  the  result  of  improving  the 
effectiveness  of  the  method  and  are  discussed  in  the  next  section.  The  analyst/researcher  noticed  that  almost  an  hour 
was  spent  identifying  the  simplest  version  of  the  task  and  distinguishing  it  from  other  possible  versions.  This  can  be 
important  for  training  purposes,  as  outlined  in  the  SCM  methodology.  However,  in  a business  context,  there  can  be 
other  purposes  for  conducting  the  heuristic  task  analysis.  The  results  of  such  an  analysis  can  be  used  to  generate  job 
aids  for  experts,  to  help  designers  structure  knowledge-management  systems,  and  for  other  purposes.  If  training  is 
not  the  primary  purpose,  then  the  analyst  might  choose  to  spend  less  time  identifying  the  simplest  version  and  other 
versions  (Step  2.  Identify  the  first  learning  episode).  Such  information  might  still  be  useful  for  distinguishing  experts 
and  novices,  even  though  sequencing  course  material  is  not  of  concern.  In  this  study,  the  analyst/researcher 
concluded  that  this  step  could  have  been  concluded  with  significantly  less  time  (approximately  20  minutes  less), 
without  diminishing  the  quality  of  the  results. 

Recommendation:  Unless  using  the  HTA  method  specifically  for  training  purposes,  perform  Step  2, 
"Identify  the  simplest  version,”  only  if  needed  to  distinguish  between  experts  and  novices  or  as  one  way  of  helping 
the  expert  access  tacit  knowledge.  As  the  expert  examines  various  instances  of  a task  in  search  of  heuristics,  it  may 
be  helpful  to  distinguish  between  simpler  and  more  complicated  versions. 

Effectiveness  of  the  HTA  method.  The  HTA  method  seemed  to  be  effective  in  its  primary  function  of 
eliciting  heuristic  knowledge  from  the  expert.  The  types  of  knowledge  delineated  in  Section  3.2  of  the  HTA  Method 
were  found  to  adequately  cover  the  range  and  types  of  task  knowledge  described  by  the  expert.  The 
analyst/researcher  did,  however,  have  problems  managing  the  two  tasks  of  classifying  the  expert’s  knowledge  and 
directing  the  interview  to  dig  deeper  into  the  expert’s  tacit  knowledge.  More  practice  with  the  methodology  should 
alleviate  this.  The  analyst/researcher  noticed  during  the  analysis  that  certain  verbs  used  by  the  expert  were  indicators 
of  tacit  knowledge.  Examples  of  these  verbs  are:  know , like,  feel,  see,  determine,  understand,  and  decide.  When  the 
analyst/researcher  asked  the  expert  why  she  liked  a certain  piece  of  art,  she  struggled  at  first  to  find  reasons,  but 
eventually  she  isolated  specific  characteristics  that  distinguished  artwork  she  liked  from  pieces  that  she  did  not  find 
acceptable. 

Recommendation:  Additional  guidance  should  be  developed  on  how  to  represent  explicit  knowledge. 
Although  this  analyst/researcher  has  only  begun  to  research  this  point,  such  guidance  could  come  from  fields  such  as 
task  analysis  or  the  expert’s  specific  field.  One  area  of  interesting  research  would  be  collaboration  between  the 
expert  and  the  analyst  to  develop  an  explicit  representation  for  knowledge  deemed  critical. 

Guidelines  for  analysts.  Throughout  the  two  interviews,  the  analyst/researcher  made  a conscious  effort  to 
avoid  academic  jargon,  and  the  expert  seemed  to  rapidly  understand  everything  the  researcher  was  saying.  In 
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moments  where  the  analyst/researcher  caught  himself  using  a technical  term,  he  laughed  it  off  with  the  expert  and 

used  the  moment  to  increase  rapport. 

Suggestions  for  Improving  the  HTA  Process 

The  following  is  a tentative  revision  of  the  HTA  process  based  on  the  findings  of  these  three  formative 

research  studies.  The  changes  and  additions  are  in  italics. 

Phase  I.  Prepare  for  Analysis  and  Design 

1 . Decide  on  a task  to  analyze  and  be  clear  about  the  reasons  for  analyzing  it. 

2.  Make  sure  you  have  enough  task  knowledge  to  have  a good  command  of  terminology  and  key  ideas. 

• Review  basic  reference  materials  and  try  to  become  familiar  with  key  concepts  and  jargon  in  the  field. 

• It  would  be  better  to  begin  by  identifying  the  simplest  version  of  the  task,  rather  than  trying  to  expand  the 
analysis  to  the  next  version. 

3.  Make  sure  you  have  enough  knowledge  about  the  uses  of  the  task  description. 

• If  the  task  description  will  be  used  primarily  for  deciding  on  the  content  and  sequence  of  instruction,  identify 
the  characteristics  of  the  learners  and  the  delivery  constraints  of  the  instruction  in  general. 

4.  Arrange  to  interview  multiple  experts. 

• Identify  at  least  2 or  3 experts  to  interview. 

• Plan  to  complete  the  analysis  with  one  expert  before  initiating  the  analysis  with  another. 

• Plan  to  interview  the  least  experienced  expert  first  and  proceed  to  interview  progressively  more  experienced 
experts  in  order. 

• Ask  one  or  more  of  the  task  experts  to  record  their  performance  of  a very  simple  version  of  the  task,  and  review 
the  recorded  material  in  advance  of  the  analysis;  or  observe  the  task  expert's  task  performance. 

5.  Prepare  in  conjunction  with  the  first  (next)  task  expert. 

• Establish  rapport  with  the  task  expert. 

• Introduce  the  HTA  method  to  the  expert. 

• Explain  basic  terms  (i.e.  guidelines,  explanatory  models,  etc.). 

6.  Prepare  for  the  interview. 

• Prepare  interview  materials  (i.e.,  index  cards  to  summarize  critical  incidents  during  task  performance). 

• Practice  the  HTA  interview  process  if  you  are  not  very  experienced  in  it. 

• Arrange  the  interview  logistics  ( e.g .,  reserve  a conference  room  where  you  can  work  without  interruptions). 

Phase  II.  Identify  the  First  Learning  Episode 

7.  Identify  the  simplest  version.  Hold  a focus  group  interview  with  multiple  task  experts,  and  help  them  to  reach 
consensus  on  the  simplest  version  of  the  task  that  is  fairly  representative  of  the  task  as  a whole.  Also  help  them 
to  describe  the  conditions  that  distinguish  that  version  from  all  other  versions. 

• You  may  want  to  use  some  other  criteria  in  addition  to  simple  and  representative. 

• It  may  be  helpful  to  have  the  expert  briefly  discuss  closely  related  tasks  and  clearly  distinguish  between  the 
main  task  and  the  related  tasks  during  the  remainder  of  the  analysis 

• Ask  the  task  experts  to  recall  the  simplest  case  they  have  ever  seen.  The  simplest  version  will  be  a class  of 
similar  cases.  Then  check  to  see  how  representative  it  is  of  the  task  as  a whole. 

• It  may  be  helpful  to  start  by  identifying  some  of  the  major  versions  of  the  task  and  the  .conditions  that 
distinguish  when  one  version  is  appropriate  versus  another. 

• Thinking  of  different  conditions  helps  to  identify  versions,  and  thinking  of  different  versions  helps  to  identify 
conditions.  Hence,  it  is  wise  to  do  both  simultaneously  (or  alternately). 

• There  is  no  single  right  version  to  choose  as  the  ’’simplest."  It  is  usually  a matter  of  trade-offs. 

• It  is  wise  to  go  through  this  process  with  several  task  experts  together  and  reach  consensus  before  going  on  to 
Step  8. 

8.  Analyze  the  organizing  content.  With  the  least  experienced  expert  you  have  not  yet  interviewed,  analyze  the 
organizing  content  (mostly  heuristics  and  descriptive  theories)  for  this  version  of  the  task. 

8. 1 Review  the  recorded  material  (or  any  other  visual  aid)  with  the  task  expert. 

8.2  Ask  the  task  expert  to  think  of  and  describe  one  specific  performance  of  the  selected  version  of  the 
task  to  focus  on  for  your  analysis,  or  ask  if  a videotaped  performance  would  be  a good  case  for  you  to 
focus  on  with  the  expert  during  the  analysis. 
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• It  is  often  helpful  to  have  a videotape  of  a typical  performance  of  the  simplest  version  of  the  task,  so 
you  and  the  task  expert  can  review  it  during  the  analysis  process,  but  asking  the  task  expert  to  recall 
one  specific  performance  and  keep  it  in  mind  throughout  the  process  is  a more  convenient  and 
inexpensive,  albeit  often  less  effective,  alternative. 

• If  you  don't  have  a videotape,  It  may  be  helpful  to  have  the  expert  describe  contextual  information 
and  particulars  of  the  specific  performance,  describing  how  the  expert  began  the  case,  how  it 
progressed  (in  sequence),  how  participants  reacted,  and  how  the  expert  dealt  with  any  problems  that 
arose. 

• It  may  be  helpful  to  prioritize  the  problems/concerns  that  arose  and  the  decisions/actions  that  the 
expert  used  to  deal  with  them. 

8.3  Decide  whether  to  use  a top-down  or  bottom-up  approach  to  analyzing  the  content.  If  top-down,  use 
Step  8.4  and  skip  Step  8.5.  If  bottom-up,  skip  Step  8.4  and  use  Step  8.5. 

8.4  If  top-down  approach,  start  by  identifying  the  general  categories  of  knowledge  that  an  expert  uses,  then 
proceed  to  analyze  each. 

• Ask  the  task  expert: 

a)  to  describe  each  decision  that  the  task  expert  made, 

b)  to  identify  the  kinds  of  knowledge  that  the  task  expert  drew  upon  to  make  the  decision,  and 

c)  to  describe  the  specific  knowledge  within  each  kind  of  knowledge  that  the  task  expert  used. 

• The  kinds  of  knowledge  are  likely  to  include:  steps,  guidelines  or  rules  of  thumb,  explanatory 
models,  descriptive  models,  and  metacognitive/decision  rules. 

• It  is  generally  helpful  to  start  by  asking  the  task  expert  if  there  are  any  steps  or  phases  of  activities 
that  are  always  performed  for  this  version  of  the  task.  If  so,  perform  a procedural  task  analysis  to 
identify  the  sequence  of  steps  and  to  see  if  any  of  those  steps  can  be  broken  down  into  substeps,  but 
those  substeps  must  be  ones  that  an  expert  thinks  of  and  uses  routinely  in  performing  that  version  of 
the  task. 

• For  guidelines,  use  the  following  process: 

1 . Identify  the  goals  for  this  specific  performance  of  the  task  under  its  conditions.  It  may  help  to 
have  the  expert  explain  the  goals  in  task-specific  terms  rather  than  in  abstract  terms  and  to  think 
of  the  goals  as  ideal  outcomes. 

2.  Identify  all  the  important  considerations  for  attaining  each  goal. 

3.  Identify  all  the  important  causal  factors  that  relate  to  each  consideration/subconsideration. 

4.  Analyze  each  causal  factor  to  identify  all  guidelines  that  an  expert  uses  to  account  for  this 
consideration. 

• For  explanatory  models,  use  the  following  guidelines: 

- For  each  guideline,  ask  the  task  expert  for  the  reasons  why  s/he  believes  it  works. 

- For  interrelated  guidelines,  you  are  likely  to  identify  a set  of  related  reasons  that  constitute  a 
causal  model  or  models. 

• For  descriptive  models,  use  the  following  guidelines: 

- Ask  the  task  expert  what  phenomena  influenced  this  particular  performance  of  the  task.  Try  to 
identify  all  causal  relationships  that  characterized  those  phenomena. 

- Be  sure  to  look  for  multiple  causes  for  each  effect  and  multiple  effects  for  each  cause.  Also  look 
for  chains  of  causes  and  effects,  and  explore  probabilities  for  each  causal  factor  to  have  each 
effect. 

• For  metacognitive/decision  rules,  use  the  following  guideline:  Find  out  what  rules  the  task  expert 
used  to  decide  when  to  use  which  steps,  guidelines,  and  descriptive  models  during  the  specific 
performance  of  the  task  being  analyzed. 

• It  is  wise  to  query  the  task  expert  about  any  of  these  kinds  of  knowledge  that  are  not  initially 
described  to  you  for  each  decision  the  task  expert  made  in  this  specific  performance  of  the  task. 

• If  the  expert  uses  words  such  as  know,  feel,  see,  understand,  like,  determine,  and  decide,  that  may  be 
an  indication  that  heuristic  knowledge  underlies  that  particular  performance. 

• It  is  often  helpful  to  periodically  ask  the  expert  some  questions  about  the  chosen  case,  to  keep  the 
analysis  focused  on  the  flow  of  that  version  of  the  task. 

• It  is  useful  to  help  the  expert  think  about  ways  the  specific  case  fell  short  of  how  it  should  have  been 
done  and  to  have  the  expert  offer  guidelines  for  how  this  specific  case  should  have  been  done. 
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• It  is  wise  to  have  some  kind  of  reference  material  to  provide  contextual  information  and  cues  and  to 
help  the  expert  be  more  precise.  During  the  iterative  interview  process,  the  visual  aid  also  helps  the 
expert  keep  on  track 

• It  may  be  helpful  to  use  index  cards  for  all  of  these  kinds  of  knowledge,  filling  them  out  with  the  task 
expert  during  the  analysis  process  with  one  piece  of  knowledge  per  card,  and  arrange  the  cards  in 
some  order  on  a table  in  front  of  both  of  you,  so  you  can  easily  switch  from  one  part  or  aspect  of  the 
task  to  another. 

8.5  If  bottom-up  approach,  ask  the  expert  to  describe  each  decision  that  s/he  made  and  the  process  through 
which  s/he  went  to  make  each  decision . 

• After  the  interview,  try  to  categorize  each  piece  of  heuristic  knowledge  according  to  these  categories:  Steps, 
guidelines  or  rules  of  thumb,  explanatory  models,  descriptive  models,  metacognitive/decision  rules. 

• Be  sure  to  "member  check"  the  interview  results  with  the  expert  in  a later  interview  to  verify/identify  the  types  of 
knowledge  underlying  each  decision. 

8.6  Ask  the  task  expert  to  think  of  similar  performances  of  the  task  that  are  within  the  realm  of  the  version 
of  the  task  you  are  currently  analyzing. 

8.7  Repeat  this  entire  process  (Steps  5 - 8.6)  with  the  next  least  experienced  task  expert  to  identify  any 
alternative  views  of  the  task  and  the  knowledge  that  underlies  its  performance. 

• For  each  more  experienced  expert,  you  should  summarize  the  previous  description  of  the  task  and  ask  the  expert 
to  review  it,  in  an  effort  to  reconcile  conflicts  and  select  among  alternative  ways  of  thinking  about  and 
performing  the  task. 

Formative  research  data  indicate  that  this  revised  HTA  process  would  have  been  more  effective  for  the 
three  cases  investigated  here.  It  remains  to  be  seen  whether  or  not  this  revised  process  will  also  work  well  for 
analyzing  other  tasks  that  have  heuristic  elements.  The  data  in  this  study  indicate  that  much  additional  guidance  is 
still  needed  for  conducting  a heuristic  task  analysis.  It  is  our  hope  that  this  study  will  encourage  others  to  conduct 
additional  research  to  improve  the  available  guidance  for  analyzing  heuristic  tasks. 
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Abstract 

Implementing  exportable  instructional  systems  has  always  been  a problem  for  instructional  developers. 
Even  the  best  instructional  systems  lose  effectiveness  when  they  are  poorly  implemented.  Researchers  have  focused 
their  efforts  on  creating  mechanisms  to  help  developers  and  managers  improve  and  control  implementation. 
However,  most  of  these  efforts  do  not  address  long-term  effects,  or  take  into  account  how  teachers  feel  and  react 
during  the  implementation  process.  This  study  presents  the  Dynamic  Implementation  Seeking  Equilibrium  (DISE) 
model,  a nonlinear  model  of  implementation  for  teacher-mediated  instructional  systems  to  measure  rate  of 
implementation  over  time.  The  DISE  model's  main  independent  variable,  the  exportability  control  factor  (ECF),  is 
an  intrinsic  quality  of  teacher-mediated  instructional  systems.  ECF  is  defined  as  the  amount  of  instructions,  support 
resources,  schedules  of  activities,  and  accountability  methods  embedded  in  the  instructional  system  to  delineate  or 
control  teachers ' actions  so  they  use  the  system  according  to  the  developers'  original  intent.  The  DISE  model 
predicts  four  distinct  scenarios:  extinction,  stable  low  implementation,  stable  high  implementation,  and  unstable 
implementation  with  catastrophic  change.  Beyond  the  implications  for  the  implementation  of  teacher-mediated 
instructional  systems,  this  model  offers  an  example  of  how  some  of  the  concepts,  techniques,  and  methodologies  of 
nonlinear  dynamics  can  be  applied  to  the  field  of  instructional  systems  development. 

Dynamic  Implementation  Seeking  Equilibrium  Model 

The  primary  purpose  of  the  Dynamic  Implementation  Seeking  Equilibrium  (DISE)  model  is  to  represent 
the  process  of  implementation  of  teacher-mediated  instructional  systems  through  time,  or,  in  other  words,  how  much 
of  a given  instructional  system  we  can  expect  teachers  to  actually  use  consistently  over  time. 


General  Characteristics  of  the  DISE  Model 

The  DISE  model  is  ..  . instead  of.  . . 

Descriptive  Prescriptive 

Dynamic  Static 

Explanatory  Evaluative 

Predictive  Reactionary 

Generalizable  Interventionist 

The  DISE  model  has  the  following  general  characteristics: 

1.  Descriptive : Descriptive  theories  show  the  effects  that  occur  when  a given  class  of  causal  events  happen, 
or  the  sequence  in  which  certain  events  take  place.  These  types  of  theories  do  not  favor  one  set  of  outcomes  over 
another;  they  do  not  prescribe  how  to  improve  or  modify  a situation.  The  DISE  model  describes  different  scenarios 
of  how  teachers  implement  teacher-mediated  instructional  systems  given  a set  of  causal  relationships.  It  does  not 
evaluate  one  scenario  to  be  preferable  to  the  others,  nor  does  it  suggest  how  to  increase  or  improve  implementation. 

2.  Dynamic:  Dynamic  models  are  those  which  take  the  time  variable  into  consideration  as  opposed  to  static 
models  which  give  one-shot  pictures  of  the  environment  or  assume  that  the  value  of  variables  does  not  change  over 
time.  The  variables  in  the  DISE  model  are  subscripted  with  the  time  variable.  Their  values  change  through  time. 

3.  Explanatory:  Explanatory  models,  a special  kind  of  descriptive  model,  attempt  to  identify  all  relevant 
components  of  a system  and  their  relationships,  to  explain  the  systemic  behavior  of  the  whole.  The  DISE  model 
attempts  to  satisfy  these  requirements  by  analytically  identifying  the  relevant  components  of  the  process  of 
implementation  and  by  representing  the  systemic  behavior  of  this  process  with  two  different  representational 
systems:  one  mathematical,  the  other  schematical. 

4.  Predictive:  Predictive  models  have  the  potential  to  describe,  to  some  degree,  a future  state  of  the  system 
under  scrutiny,  given  certain  present  values.  The  DISE  model  can  be  used  to  predict  future  rates  of  implementation 
of  teacher-mediated  instructional  systems  (the  dependent  variable)  by  estimating  the  present  values  of  the 
independent  variables. 

5.  Generalizable:  Generalizable  theories  apply  to  any  situation  within  the  model’s  scope.  This  characteristic 
is  a matter  of  degree  and  limited  by  the  scope  of  the  theory  under  development.  The  DISE  model  has  been 
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successfully  applied  to  a variety  of  teacher-mediated  instructional  systems.  However,  further  research  is  necessary  to 
ascertain  the  degree  of  generalizability  of  the  model. 

Roles  of  Those  Involved  in  the  Implementation  Process 

Before  being  able  to  delimit  the  scope  of  this  study  clearly,  it  is  necessary  to  define  the  roles  of  all  the 
participants  in  the  implementation  process.  These  roles  will  help  delimit  the  scope  of  the  model  in  the  next  section. 

Developers.  A class  of  instructional  designers  who  generate  explicit  objectives,  validated  tests,  and 
materials  that  are  instructional  and  replicable  ( materials  developers  according  to  Burkman,  1987)  and  whose  aim  is 
to  have  a direct  influence  on  the  behavior  of  individual  learners.  However,  in  the  context  of  the  DISE  model,  these 
developers  produce  information  for  a cluster  of  learners  who  meet  at  a specified  time,  led  by  a teacher  who  controls 
what  occurs  in  the  classroom  ( micro-instructional  designers  according  to  Burkman,  1987). 

Teachers.  Those  potential  adopters  of  developers’  instructional  innovations  who  actually  use  these 
instructional  materials,  leading  a group  of  learners  to  achieve  a certain  learning  outcome.  Teachers  are  ultimately  in 
charge  of  deciding  what,  when,  and  how  to  implement  the  new  instructional  system.  This  role  is  defined  in  a broad, 
generic  sense  referring  to  the  person(s)  mediating  the  instruction;  they  could  be  teachers  in  traditional  classroom 
settings,  professors,  instructors,  trainers,  tutors,  etc.  Also,  depending  on  the  scale  at  which  the  instructional  system  is 
being  analyzed,  the  role  of  teacher  could  refer  to  an  individual  teacher,  or  an  aggregate  of  teachers,  such  as  a 
department,  a school,  a military  or  industrial  training  unit,  or  an  entire  school  district. 

Learners.  Those  potential  adopters  who  are  the  ultimate  audience  and  target  of  the  instructional  system. 
Learners  should  benefit  from  the  instructional  system  by  obtaining  new  knowledge  or  skills  according  to  the 
system’s  learning  outcomes. 

Master  trainers.  Some  instructional  systems  require  teachers  undergo  some  sort  of  initial  training  or 
certification  training  before  they  are  qualified  or  certified  to  start  teaching  those  particular  instructional  systems. 
Those  teachers  who  teach  this  type  of  certification  training  to  other  teachers  are  called  in  this  study  master  trainers. 
Their  role  is  important  because,  even  though  developers  may  have  included  detail  instructions  on  how  to  “correctly” 
deliver  the  instructional  system’s  initial  training,  it  still  is  their  prerogative  to  deliver  this  training  according  to  their 
own  interpretations. 

Scope  Delimitation 

The  primary  purpose  of  the  DISE  model  is  to  represent  the  process  of  implementation  of  teacher-mediated 
instructional  systems  through  time,  or,  in  other  words,  how  much  of  a given  instructional  system  we  can  expect 
teachers  to  actually  use  consistently  over  time.  This  purpose  statement  implies  two  scope  limitations  about  the  kind 
of  instructional  system  under  scrutiny:  (a)  The  instructional  system  has  to  be  exported,  and  (b)  it  has  to  be  teacher- 
mediated. 

Exportability  means  that  the  instructional  system  is  designed  to  be  used  by  a group  of  teachers  different 
from  those  developers  who  created  the  system.  To  make  an  instructional  system  exportable,  developers  purposefully 
create  devices  to  communicate  their  idea  of  how  the  system  should  be  used  to  those  teachers  who  will  implement  the 
system.  This  signifies  that  implementation  starts  once  developers  and  change  agents  in  charge  of  training  teachers 
and  promoting  the  adoption  of  the  innovation  leave  the  scene  and  teachers  are  not  under  the  direct  influence  of  those 
who  created  or  promoted  the  instructional  system. 

Teacher-mediated  instructional  systems  require  a teacher  to  present  the  content  to  the  learner.  Teacher- 
mediated  instructional  systems  pose  a special  problem  for  developers.  Developers  must  not  only  arrange  the  content 
for  learners,  but  also  guide  the  teachers’  actions  to  present  this  content  appropriately.  To  do  this,  developers  create  a 
schema  of  the  instructional  environment,  delineating  interactions,  schedules,  activities,  etc.  Teachers  also  have  a 
special  challenge  when  using  teacher-mediated  instructional  systems;  they  have  to  not  only  understand  the  content, 
but  also  interpret  and  implement  the  developers’  wishes  on  how  to  present  the  content  to  learners. 

For  example,  a self-study  workbook  is  exportable  but  not  teacher-mediated,  so  it  is  outside  the  scope  of  this 
study.  On  the  other  hand,  an  instructor’s  manual  for  a class  that  teaches  word-processing  is  both  exportable  and 
teacher-mediated,  so  it  would  be  subject  to  the  analysis  of  this  model. 

Identified  Assumptions 

In  this  section  I discuss  the  necessary  assumptions  which  I have  been  able  to  identify.  I will  explain  each 
assumption.  These  assumptions  are  necessary  to  guide  the  modeling  process  while  building  the  DISE  model. 
Because  they  are  assumed,  these  propositions  are  accepted  as  valid;  however,  their  validity  should  be  tested  in  future 
research  efforts. 
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Teachers  seek  equilibrium.  This  assumption  is  based  on  experience,  observation,  and  reflection  upon 
human  nature.  People  in  general  strive  to  find  equilibrium  in  their  lives.  Just  as  physical  systems  tend  toward  a state 
of  equilibrium,  teachers  seem  to  have  a psychological  tendency  to  seek  after  a state  of  equilibrium  in  their  teaching 
practices.  Teachers  in  particular  seek  to  teach  in  a personal,  uniform  way.  When  teachers  change  the  way  they  teach, 
they  do  so  in  order  to  improve  their  teaching,  to  polish  their  techniques,  to  find  better  examples  and  explanations. 
Those  practices  that  work  they  keep  and  use  the  next  time.  Those  that  do  not,  they  change,  seeking  better  ones.  All 
these  changes  are  done  cautiously  and  consciously  as  they  seek  to  achieve  a state  of  equilibrium  where  they  feel 
satisfied  with  the  learning  taking  place  in  their  classes.  Granted,  this  state  is  elusive  and  ill  defined,  but  can  be 
observed  when  a teacher  teaches  certain  contents  the  same  as  in  previous  experiences.  The  DISE  model  assumes  that 
teachers  seek  this  state  of  equilibrium,  and  when  they  do  not  achieve  it,  they  feel  increasingly  frustrated  in  an 
unstable,  ever-changing  environment. 

Exogenous  changes,  such  as  having  to  implement  a new  instructional  system,  are  disruptive  to  this 
equilibrium.  Teachers  seek  to  quickly  adopt  and  adapt  the  features  of  the  new  instructional  system  and  reach  a new 
level  of  equilibrium.  Teachers  may  only  introduce  changes  to  their  teaching  or  are  willing  to  try  new  things  when 
confronted  with  new  content  or  problems;  however,  once  they  find  some  set  of  teaching  practices  that  work  for  them 
and  their  students,  they  often  prefer  to  maintain  them  and  use  them  time  after  time  with  little  or  no  change. 

Constant  quality.  Because  the  focus  of  the  study  is  on  implementation,  it  assumes  that  the  instructional 
system’s  quality  is  constant.  In  other  words,  the  effectiveness  of  the  instructional  system  is  not  under  scrutiny. 

Neutral  value  of  implementation.  The  DISE  model  attempts  to  avoid  the  pro-innovation  bias  (Rogers, 
1995,  p.  100)  by  describing  the  implementation  process  impartially,  without  assuming  that  higher  levels  of 
implementation  are  better,  or  more  desirable,  than  lower  ones.  This  study  considers  rejection,  partial 
implementation,  high  levels  of  implementation,  and  anything  in  between  as  likely  outcomes  of  the  implementation 
process.  Therefore,  this  study  assumes  not  only  that  the  quality  of  the  instructional  system  is  high  and  constant,  but 
also  that  if  teachers  decide  not  to  implement  the  instructional  system  or  to  implement  something  else  instead, 
learning  outcomes  are  not  affected. 

Teacher,  not  learner  implementation.  The  DISE  model  analyzes  implementing  the  instructional  system 
from  the  teacher’s  point  of  view,  without  taking  into  account  what  learners  choose  to  do.  For  example,  if  the  teacher 
asks  learners  to  complete  certain  worksheets,  this  study  considers  those  worksheets  implemented  even  if  learners 
choose  not  to  complete  them. 

Teachers  try  their  best.  This  assumption  implies  that  teachers  eagerly  strive  to  do  their  best  in  deciding 
what  to  teach  and  how  to  teach,  putting  the  learners’  instructional  interest  first.  They  do  so  not  just  because  it  is  a 
job,  but  because  they  maximize  their  personal  enjoyment  when  teaching  at  their  best.  This  assumption  removes  any 
second-guessing  on  the  motives  for  teachers’  deciding  for  or  against  implementing  certain  parts  of  the  instructional 
system.  The  DISE  model  does  not  take  into  account  other  types  of  motives. 

Teachers  are  free  implementation  agents.  Teachers  are  assumed  to  be  relatively  free  and  independent 
and  can  choose  how  they  are  going  to  teach  their  classes.  This  assumption  recognizes  that  teachers  are  influenced  by 
the  environment  in  which  they  work  to  make  implementation  decisions;  nevertheless,  once  the  classroom  door  is 
closed,  teachers  are  basically  free  to  conduct  their  class  however  they  want.  Academic  freedom  is  one  of  the  most 
cherished  and  defended  principles  of  the  educational  system.  Teachers  endure  many  things,  even  shamefully  low 
remunerations,  in  order  to  enjoy  academic  freedom. 

Teachers  are  creative.  One  of  the  reasons  many  teachers  teach  is  to  satisfy  personal  psychological  needs. 
Chief  among  these  are  the  needs  to  feel  creative  and  needed.  Of  course  these  needs  vary  in  nature  and  intensity  from 
individual  to  individual,  but  teachers  have  to  satisfy  these  psychological  needs;  otherwise,  a continuous  repressed 
creativity  generates  feelings  of  frustration.  In  this  study,  the  expression  “creativity  need”  is  used  as  a general  way  to 
encompass  all  the  personal  psychological  needs  teachers  seek  to  satisfy  by  teaching.  To  satisfy  their  creativity, 
teachers  experiment  in  their  classes  with  different  teaching  techniques  and  strategies,  which  should  be  their  own 
creation,  until  they  feel  that  their  students  are  “getting  it.”  When  teachers  are  told  what  to  do,  even  if  it  is  something 
good  for  the  students’  learning,  it  does  not  satisfy  their  creativity.  In  fact,  it  may  make  them  feel  unneeded, 
superfluous,  that  anyone  could  do  it.  Some  of  the  ways  teachers  express  their  creativity  is  by  lesson  planning, 
developing  teaching  materials,  and  solving  instructional  situations  through  previous  experience  and  improvisation 
when  necessary. 

Teachers  feel  frustrated  when  they  cannot  express  their  creativity.  As  a corollary  of  the  previous 
assumption,  it  is  further  assumed  that  teachers  feel  frustrated  when  having  to  repeatedly  repress  their  creativity  by 
having  to  implement  very  controlling  or  highly  prescriptive  instructional  systems.  For  example,  if  the  instructional 
system  prescribes  every  interaction  between  teachers  and  learners,  it  would  leave  little  or  no  opportunity  for  teachers 
to  express  their  creativity.  In  this  hypothetical  situation,  teachers  would  have  to  decide  whether  to  comply  with  the 
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instructional  system  and  repress  their  creative  feelings  or  implement  less  and  make  some  room  to  express  their  own 
creativity.  If  teachers  find  that  they  have  to  repress  their  creativity  often,  it  is  assumed  that  these  feelings  of 
repressed  creativity,  of  disequilibrium,  of  being  controlled,  will  lead  to  teachers’  feeling  frustration  toward  the 
instructional  system. 

Transfer  of  training  research  applies  to  teachers  too.  The  DISE  model  assumes  that  when  teachers 
mediate  the  instruction  as  they  implement  teacher-mediated  instructional  systems,  they  need  to  transfer  the 
information  to  their  own  students.  This  implies  that  the  transfer  of  training  research  findings  (Bransford,  1979;  Clark 
& Voogel,  1986;  Cronbach  & Snow,  1977;  Mayer,  1980)  apply  to  this  study.  The  transfer  of  training  research  will 
be  used  to  construct  one  of  the  variables  of  the  DISE  model. 

Developers  have  a specific  implementation  schema.  The  DISE  model  assumes  that  developers  have 
specific  ideas  or  a schema  of  how  the  instructional  system  should  be  implemented.  Even  though  developers  may  not 
be  fully  cognizant  of  their  own  implementation  schema  or  may  not  fully  represent  it  in  their  design,  this  schema 
exists  and  it  corresponds  to  the  way  the  instructional  developers  would  use  their  own  instructional  system  if  they 
were  teaching.  Developers  express  these  ideas  not  only  by  arranging  the  content  for  learners,  but  also  by  guiding 
teachers’  actions  to  present  this  content  appropriately.  To  more  effectively  communicate  their  schema  and  increase 
exportability,  instructional  developers  carefully  delineate  instructions,  provide  resources,  schedule  activities,  and 
create  methods  of  accountability. 

This  assumption  is  necessary  because  the  implementation  of  this  schema  is  equal  to  100  percent 
implementation.  If  a teacher  were  able  to  use  the  instructional  system  exactly  the  same  way  that  the  developer 
would,  then  this  teacher  would  be  perfectly  implementing  the  system. 

Perfect  implementation  is  unattainable.  An  assumed  corollary  from  the  previous  assumption  is  that 
teachers  are  never  quite  able  to  perfectly  capture  the  developers’  schema.  Because  of  personal  differences, 
communication  problems,  and  previous  experiences,  teachers  cannot  attain  100  percent  implementation.  This 
assumption  is  necessary  to  set  the  range  of  the  main  independent  variable  of  the  DISE  model. 

This  section  listed  the  assumptions  I was  able  to  identify  as  necessary  for  building  the  DISE  model.  The 
next  section  defines  the  variables  of  the  model. 

Variables  of  the  Model 

Rate  of  implementation  (/).  One  way  to  assess  implementation  in  general  is  to  measure  the  rate  of 
implementation  (7).  This  is  the  main  dependent  variable  of  the  DISE  model.  It  shows  what  percentage  of  the  original 
instructional  system  is  being  used  at  each  point  in  time.  A value  close  to  1 shows  that  teachers  are  implementing 
almost  everything  the  original  developers  intended  them  to  do,  that  most  of  the  instructional  system  is  being  utilized. 
A rate  close  to  0 shows  that  teachers  are  utilizing  only  a small  portion  of  the  original  instructional  system. 

By  choosing  to  measure  rates  of  implementation,  instead  of,  let’s  say,  actual  implementation,  the  DISE 
model  applies  to  a variety  of  settings  and  permits  for  comparisons  among  dissimilar  instructional  systems — the 
model  is  generalizable.  However,  this  choice  also  implies  that  the  model  does  not  specify  what  parts  of  the 
instructional  system  will  be  implemented  and  which  ones  will  be  discarded  by  teachers,  but  it  only  states  how  much 
will  be  implemented.  For  example,  two  teachers  could  implement  the  same  instructional  system  at  the  same  rate  but 
choose  to  use  different  parts  of  the  instructional  system. 

Initial  rate  of  implementation.  The  initial  value  of  I (I0)  represents  the  rate  of  implementation  after  initial 
training,  just  as  the  instructional  developers  pull  away  from  the  scene — when  the  transfer  of  the  instructional  system 
actually  occurs.  If  developers  do  a good  job  training  teachers  in  using  the  new  instruction,  then  I0  would  be  high. 
When  developers  transfer  the  new  instructional  system  with  little  or  no  effort  to  inform  teachers  how  to  use  it,  the 
starting  implementation  value  would  be  low. 

Time  iterations  ( t ).  The  subscript  (/,  / + 1,  t + 2,  / + n ),  introduced  in  the  previous  paragraphs,  refers  to 

how  many  times  teachers  have  used  the  system.  Thus,  t is  an  ordinal,  discrete  variable  instead  of  a real,  continuous 
one.  In  the  present  context,  this  makes  sense  because  teachers  are  not  constantly  engaged  in  the  instructional 
process;  instead  they  engage  in  it  at  intervals.  Sometimes  these  intervals  are  very  frequent  and  regular  (every  day,  or 
even  every  shift);  other  times  the  intervals  are  over  longer  periods  of  time  or  irregular  (semester,  school  year,  or 
training  cycle).  The  length  of  the  intervals  between  one  iteration  and  the  next  also  depends  on  the  scale  at  which  we 
are  analyzing  the  system  (an  individual  teacher,  a department,  a school,  or  an  entire  school  district).  In  the  DISE 
model,  every  variable  that  changes  through  time  will  have  this  subscript. 

Complementary  activities  ( /-/, ).  Another  component  of  teachers’  implementation  experience  is  7,’s 
complement.  All  the  instructional  activities  teachers  perform  during  one  iteration  can  be  thought  of  as  a whole  (7). 
Those  activities  not  prescribed  as  part  of  the  instructional  system  which  teachers  nevertheless  purposefully  perform, 
and  which  yield  a perceived  positive  result,  are  here  called  complementary  activities  (/  - 7, ).  It  is  recognized  that 
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this  is  a pseudo-variable  because  it  is  defined  in  terms  of  rate  of  implementation;  meaning  that  it  is  defined  by  what 
the  rate  of  implementation  is  not.  This  factor  represents  the  successful  activities  teachers  implement  in  the  learning 
situation  that  do  not  originate  from  the  instructional  system.  Another  way  of  looking  at  this  factor  is  as  an  expression 
of  teachers’  need  to  fulfill  feelings  of  creativity  and  usefulness. 

Exportability  Control  Factor  (ECF,  Kt  in  the  equations).  As  mentioned  previously,  developers  strive  to 
communicate  their  internal  schema  of  the  instructional  system.  However,  without  direct  contact,  all  developers  can 
do  is  to  train  teachers,  specify  precise  instructions,  create  resources,  dictate  schedules,  and  enforce  accountability 
from  teachers;  in  short,  they  can  develop  an  exportable  instructional  system.  The  aim  of  these  exportability  devices 
is  to  communicate  the  developers’  intentions  of  how  the  instruction  should  work.  The  amount  and  specificity  of 
these  exportability  devices  constitute  an  intrinsic  quality  of  teacher-mediated  instructional  systems,  hereby  called 
Exportability  Control  Factor  (ECF,  in  the  equations,  is  represented  by  the  letter  K).  This  name  suggests  the  amount 
and  specificity  of  control  devices  contained  in  the  instructional  system  to  ensure  its  exportability. 

This  new  theoretical  construct  has  a decisive  influence  on  implementation.  In  fact,  if  a teacher-mediated 
instructional  system  is  going  to  be  truly  exported,  developers  cannot  have  any  contact  with  teachers  after  they 
receive  the  instructional  system;  developers  can  only  influence  implementation  by  designing  the  system’s  ECF 
(through  instructions,  support  resources,  schedules  of  activities,  and  accountability  methods)  and  the  initial  rate  of 
implementation,  I0  (through  initial  training.)  Therefore,  ECF  is  defined  as  the  (a)  initial  training,  (b)  instructions,  (c) 
support  resources,  (d)  schedule  of  activities,  and  (e)  accountability  methods  embedded  in  the  instructional  system  to 
delineate  or  control  teachers’  actions  so  the  instructional  system  is  implemented  according  to  the  developers’ 
original  intent. 

For  example,  the  ECF  of  an  instructional  system  is  high  if  it  specifies  each  step  teachers  need  to  take  in 
order  to  continue.  A medium  ECF  would  be  a system  which  gives  suggestions,  explicit  ideas,  or  a choice  of 
exercises  for  teachers  to  pick  and  use  or  modify.  On  the  other  extreme,  an  instructional  system  with  a low  ECF 
contains  broad  suggestions  of  possible  courses  of  action  teachers  should  take,  or  visionary  statements  of  the  ideal 
state  of  affairs  with  few  or  no  specifics. 

1 .  Initial  training 

2.  Instructions 

3.  Support  resources 

4.  Schedule  of  activities 

5.  Accountability  methods 

Transfer  Alignment  Coefficient  (7).  Before  defining  this  variable,  it  is  necessary  to  define  two  related 
concepts — expected  transfer  and  elicited  transfer. 

Expected  transfer  is  the  type  of  transfer  (near  or  far)  necessary  to  effectively  transfer  different  types  of 
learning  outcomes  (procedural  or  declarative).  For  example,  the  expected  transfer  for  a declarative  learning  outcome 
is  far  transfer.  The  more  conceptual  the  learning  objective,  the  farther  the  expected  transfer  would  be;  conversely, 
the  more  procedural  the  learning  objective,  the  nearer  the  expected  transfer  would  be. 

Elicited  transfer  is  the  type  of  transfer  (near  or  far)  most  likely  to  be  achieved  by  the  type  of  instructional 
strategy  (behavioral  or  cognitivist)  the  instructional  system  prescribes  teachers  to  use  for  teaching  different 
outcomes.  For  example,  if  the  instructional  system  uses  cognitive  strategies  for  the  teacher  to  teach,  the  elicited 
transfer  is  far.  The  more  cognitive  or  constructivist  the  instructional  strategy  of  the  system,  the  farther  the  elicited 
transfer  would  be;  conversely,  the  more  behavioral  the  instructional  strategy,  the  nearer  the  elicited  transfer  would 
be. 

Therefore,  transfer  alignment , T,  is  the  correlation  coefficient  (-1  < T < 1)  between  the  expected  and  the 
elicited  transfer  per  learning  objective.  If  the  instructional  strategies  used  by  the  instructional  system  to  teach 
different  types  of  learning  objectives  elicit  the  same  type  of  transfer  as  would  be  expected  for  the  type  of  objective, 
then  the  transfer  alignment  would  be  strong  (close  to  1,  aligned).  If  there  is  no  relationship  between  the  chosen 
instructional  strategies  and  the  type  of  objectives  (for  example,  a uniform  behavioral  approach  is  used  to  teach  all 
sorts  of  learning  objectives),  transfer  alignment  would  be  null  (close  to  0,  unaligned).  If  the  wrong  strategy  is  used  to 
teach  the  objectives  or  to  teach  the  teachers  how  to  teach  these  objectives  (as  in  the  case  when  the  instructional 
system  behaviorally  prescribes  for  instructors  how  to  teach  cognitivist  strategies  to  achieve  cognitivist  outcomes), 
transfer  alignment  would  be  negatively  correlated  (close  to  -1,  misaligned). 

For  example,  if  the  instructional  system  deals  with  procedural  learning  outcomes  and  uses  behavioral 
teaching  methods,  then  teachers  only  need  to  achieve  near-transfer  of  the  instruction  to  their  learners.  The  type  of 
learning,  teaching  methodology,  and  transfer  required  are  aligned. 

On  the  other  hand,  if  the  instructional  system  deals  with  declarative  and  conditional  learning  outcomes  but 
uses  behavioral  teaching  methods,  or  the  system  behaviorally  requires  teachers  to  use  cognitive  methods,  then 
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teachers  may  only  be  able  to  achieve  near-transfer  of  the  instruction  to  the  learners.  Simultaneously,  they  must 
realize  that  the  system  is  inadequate  in  achieving  the  level  of  far-transfer  required  to  meet  the  declarative  and 
conditional  learning  outcomes.  There  is  a misalignment  between  the  type  of  learning,  teaching  methodology,  and 
transfer  required. 

For  example,  if  the  instructional  system  deals  with  procedural  learning  outcomes  and  uses  behavioral 
teaching  methods,  then  teachers  only  need  to  achieve  near-transfer  of  the  instruction  to  their  learners.  The  type  of 
learning,  teaching  methodology,  and  transfer  required  are  aligned. 

On  the  other  hand,  if  the  instructional  system  deals  with  declarative  and  conditional  learning  outcomes  but 
uses  behavioral  teaching  methods,  or  the  system  behaviorally  requires  teachers  to  use  cognitive  methods,  then 
teachers  may  only  be  able  to  achieve  near-transfer  of  the  instruction  to  the  learners.  Simultaneously,  they  must 
realize  that  the  system  is  inadequate  in  achieving  the  level  of  far-transfer  required  to  meet  the  declarative  and 
conditional  learning  outcomes.  There  is  a misalignment  between  the  type  of  learning,  teaching  methodology,  and 
transfer  required. 

Accumulated  Frustration  (At).  Whenever  teachers  increase  the  rate  of  implementation  of  the  instructional 
system  from  one  interaction  to  the  next  ( It+1  > /,,  the  rate  of  implementation  now  is  greater  than  that  during  the 
previous  iteration,)  they  have  to  sacrifice  some  of  their  freedom  to  use  their  own  complementary  activities  and 
repress  their  creativity.  This  repressed  creativity  generates  frustration,  which  accumulates  throughout  the 
implementation  process.  The  amount  of  accumulated  frustration  is  a function  of  (a)  the  personal  degree  of  need  to  be 
creative,  (b)  the  tolerance  to  repress  this  creativity,  and  (c)  the  difference  in  rates  of  implementation  (7,+/  - /,.) 
whenever  this  difference  is  positive.  Only  upward  changes  in  the  rate  of  implementation  accumulate  frustration. 
When  teachers  suppress  their  creativity  in  order  to  increase  implementation,  (It+I  - IJ  > 0,  frustration  accumulates 
proportionally  to  this  increase.  It  is  assumed  that  when  implementation  decreases,  (7,+/-/^  < 0 , frustration  remains 
static.  The  definition  of  this  variable  is  based  on  the  previously  stated  assumption  that  teachers  feel  frustrated  when 
they  cannot  express  their  creativity. 

The  six  variables  discussed — rate  of  implementation  (/,),  time  iterations  (/),  complementary  activities  (7  - 
7, ),  exportability  control  factor  (ECF,  K ),  transfer  alignment  (7),  and  accumulated  frustration  (A) — are  used  to  model 
the  dynamic  process  of  implementation  for  teacher-mediated  instructional  systems.  Next,  the  relationships  among 
these  variables  are  presented  to  build  the  DISE  model. 

Relationships  among  Variables 

Implementation  as  a function  of  Exportability  Control  Factor.  By  definition,  developers  design  their 
instructional  systems  with  a certain  degree  of  ECF.  This  depends  on  the  amount  and  specificity  of  initial  training, 
instructions,  support  resources,  schedule  of  activities,  and  accountability  methods  included  in  the  system.  The  main 
purpose  of  ECF  is  to  ensure  exportability,  to  exercise  some  sort  of  pressure  to  implement  the  instructional  system. 
This  pressure  influences  teachers’  decisions  on  how  much  of  the  instructional  system  to  implement,  and  how  much 
of  their  own  complementary  activities  to  include.  This  relationship  is  represented  by  Equation  1 (All  the  equations 
are  found  in  Figure  1 at  the  end  of  this  document.) 

Implementation  as  a function  of  previous  Implementation.  One  of  the  most  powerful  influences  on  a 
teacher’s  future  rate  of  implementation  is  that  teacher’s  previous  implementation  experience  with  the  instructional 
system  itself.  Most  teachers  are  constantly  monitoring  their  own  teaching  practice,  looking  for  those  things  that 
work  to  repeat  them  in  the  future,  and  identifying  those  things  that  fail  to  avoid  them.  The  way  they  implemented  the 
system  the  previous  time  ( t ) they  used  it  is  going  to  strongly  influence  the  rate  of  implementation  of  the  system  at 
f+7.  For  example,  the  way  a teacher  uses  a particular  instructional  system  during  today’s  science  class  will  largely 
depend  on  the  perceived  results  of  the  previous  time  the  teacher  used  the  system,  probably  the  day  before  during  the 
science  period.  See  Equation  2. 

Implementation  as  a function  of  previous  Complementary  Activities.  Another  force  influencing  a 
teacher’s  future  rate  of  implementation  is  the  effect  of  previously  implemented  complementary  activities.  Those 
activities  not  prescribed  as  part  of  the  instructional  system  which  teachers  nevertheless  purposefully  perform,  and 
which  yield  a perceived  positive  result,  the  complementary  activities,  tend  to  lower  the  rate  of  implementation  the 
next  time  teachers  use  the  instructional  system.  This  is  represented  by  Equation  3. 

Combining  all  three  relationships.  The  discussion,  so  far,  posits  that  future  rates  of  implementation  are  a 
function  of  previous  implementation,  previous  use  of  complementary  activities,  and  ECF.  Now,  these  three  are 
combined  in  a logical  manner. 

As  previously  stated,  the  assumption  has  been  made  that  teachers  seek  to  achieve  a state  of  equilibrium  in 
their  teaching  practices.  Thus,  when  faced  with  the  implementation  of  new  instructional  materials,  teachers  will  seek 
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to  integrate  the  new  materials  with  their  own  instructional  practices  until  they  achieve  a state  of  equilibrium.  They 
will  try  different  mixes  of  components  of  the  instructional  system  and  complementary  activities  until  they  are 
satisfied  with  the  new  way  of  teaching.  This  process  takes  some  time;  implementation  changes  through  each 
iteration  until  the  new  equilibrium  is  achieved.  The  equilibrium-seeking  process  is  represented  by  the  balancing 
feedback  loop  at  the  top  of  Figure  2.  The  delay  in  the  feedback  loop  represents  the  t variable,  the  time  interval 
between  each  iteration  when  teachers  evaluate  their  previous  implementation  experience  which  influences  their  next 
implementation. 

Equation  4 is  not  only  the  simplest  model,  but  also  one  that  appropriately  combines  the  logical  relationships 
among  the  factors  affecting  implementation,  including  ECF.  It  shows  how  /,+/  is  proportional  to  the  product  of  /,  and 
(1  -/,).  Equation  4 should  be  interpreted  as  follows:  “The  rate  of  implementation  for  the  next  time  the  teacher  uses 
the  instructional  system  depends  on  the  interactions  among  the  exportability  control  factor  of  the  system,  the  current 
rate  of  implementation,  and  the  current  amount  of  complementary  activities/*  This  equation  is  one  commonly 
referred  to  in  discussions  of  chaos  theory,  and  it  has  been  applied  to  model  a variety  of  settings  including  population 
growth  (May,  1976),  population  growth  of  urban  areas,  economic  growth  limited  by  technology,  city  development 
limited  by  existing  infrastructure  (Cartwright,  1991),  and  organization  of  social  environments  (Marion,  1992). 

Graphically,  this  equation  is  represented  by  the  balancing  loop  between  /,  and  (1  - /,)  and  the  external 
influence  of  ECF  diagramed  in  Figure  2. 

The  effects  of  Exportability  Control  Factor  and  initial  implementation  on  the  final  rate  of 
implementation.  The  mathematical  representation  of  the  DISE  model,  thus  far,  affords  the  opportunity  to  analyze 
the  model’s  behavior  from  different  perspectives.  For  example,  with  Equation  4 the  behavior  of  how  /,  changes  over 
time  can  be  analyzed.  As  mentioned  earlier,  instructional  developers  can  only  influence  ECF  (directly  through 
instructions,  support  resources,  schedules  of  activities,  and  accountability  methods)  and  the  initial  rate  of 
implementation,  I0  (indirectly  through  initial  training.)  However,  according  to  Equation  4,  no  matter  what  the  value 
of  Iq  is,  teachers  would  tend  to  implement  the  instructional  system  at  about  the  same  rate  after  some  time.  The  long- 
term implementation  of  teacher-mediated  instructional  systems  is  determined  by  the  system’s  ECF  and  not  by  the 
initial  rate  of  implementation. 

This,  however,  should  not  be  construed  as  implying  that  these  teachers  would  implement  the  system  exactly 
the  same  way.  The  DISE  model  only  shows  rates  of  implementation,  but  it  does  not  specify  what  parts  of  the 
instructional  system  will  be  implemented  and  which  ones  will  be  discarded  by  teachers.  Also,  the  complementary 
activities,  which  in  these  cases  are  about  one-third  of  what  the  three  teachers  do  in  their  classrooms,  are  most  likely 
going  to  be  different. 

The  effect  of  Time  on  Exportability  Control  Factor.  As  mentioned  earlier,  ECF  is  an  intrinsic  quality  of 
instructional  systems  and  as  such  it  is  constant;  however,  teachers  perceive  this  quality  differently  depending  on 
several  factors.  Among  these,  time  is  an  important  factor  which  changes  a teacher’s  perception  of  ECF.  For 
example,  as  teachers  gain  experience  with  the  instructional  system,  they  become  more  familiar  with  it  and  feel  more 
comfortable  with  it,  thus,  their  perceived  ECF  should  decrease  slightly.  In  order  to  account  for  these  changes,  the 
constant  ECF  (an  intrinsic  quality  of  the  instructional  system)  should  be  changed  to  a variable  ECF  (the  exportability 
control  of  the  system  as  perceived  by  a teacher  at  a point  in  time).  The  constant  ECF  could  be  used  as  a predictive 
tool  to  guide  the  design  process.  The  variable  ECF  should  be  used  as  an  explanatory  tool  during  the  actual 
implementation  process.  Therefore,  to  model  the  changes  in  perceived  ECF,  a t subscript  should  be  added  to  this 
variable,  as  shown  in  Equation  4.  Within  the  context  of  the  DISE  model,  I call  Equation  4 the  implementation 
equation. 

The  effect  of  Transfer  Alignment  Coefficient  on  Exportability  Control  Factor.  Now  the  attention  is 
turned  to  the  next  variable  of  the  DISE  model,  transfer  alignment  coefficient,  T.  What  would  be  the  effect  of  the 
transfer  alignment  coefficient  on  perceived  ECF,  Kf! 

It  is  theorized  that  an  aligned  instructional  system,  one  with  a positive  transfer  alignment  coefficient,  would 
have  the  effect  of  reducing  the  perceived  ECF  because  teachers  would  feel  more  confident  on  the  instructional 
soundness  of  the  system.  Even  highly  prescriptive  instructional  systems,  with  ECF  < 3,  would  seem  less  controlling 
when  the  system’s  instructional  strategies  are  aligned  with  its  learning  objectives. 

Conversely,  a misaligned  instructional  system,  one  with  a negative  transfer  alignment  coefficient,  would 
increase  the  perceived  ECF  because  teachers  would  feel  confused  by  not  being  able  to  efficiently  transfer  the 
instructions  found  in  the  system  to  their  teaching  situations. 

It  is  further  theorized  that  unaligned  instructional  systems,  transfer  alignment  coefficients  close  to  zero, 
would  have  little  or  no  effect  on  perceived  ECF.  This  unalignment  would  just  solidify  teachers’  perceived  ECF. 

A simple  way  to  model  this  relationship  is  to  subtract  the  transfer  alignment  coefficient  from  the  perceived 
ECF,  (Kt  - T).  This  relationship  is  shown  by  Factor  5 in  Figure  1 . 
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The  effect  of  Transfer  Alignment  Coefficient  on  Complementary  Activities.  Furthermore,  transfer 
alignment  coefficient  also  has  an  effect  on  how  teachers  evaluate  the  effect  of  complementary  activities.  Positive 
transfer  alignment  coefficients  would  strengthen  the  influence  of  previous  implementation,  /„  over  complementary 
activities,  (1  - IJ  because  aligned  instructional  strategies  would  be  more  appealing  to  continue  implementing. 
Conversely,  negative  transfer  alignment  coefficients  would  strengthen  the  influence  of  complementary  activities, 
( 7- ),  over  previous  implementation,  /,,  because  in  these  situations  teachers  would  feel  more  justified  to  continue 
implementing  their  own  activities. 

The  DISE  model  mathematically  represents  this  relationship  by  dividing  the  complementary  activities 
factor,  (1  - IJ,  by  (1  + T/2 ),  showing  that  a positive  transfer  alignment  would  lower  the  pressure  to  implement 
complementary  activities.  This  relationship  is  modeled  in  Factor  6. 

From  my  limited  experience  analyzing  instructional  systems,  I suspect  that  most  instructional  systems  are 
unaligned  ( T close  to  zero)  because  developers  would  tend  to  apply  similar  instructional  strategies,  those  with  which 
they  feel  most  comfortable,  to  address  different  types  of  learning  outcomes.  For  example,  behaviorist  instructional 
developers  are  likely  to  apply  behaviorist  learning  strategies  for  all  types  of  learning  outcomes,  be  they  procedural, 
declarative,  or  conditional.  The  same  could  be  suspected  of  cognitivists  and  constructivists.  I also  forward  that 
aligned  instructional  systems  ( T close  to  1)  are  more  likely  to  be  those  systems  which  have  been  carefully  designed, 
paying  close  attention  to  address  learning  outcomes  with  appropriate  instructional  strategies.  This  implies  that 
transfer  alignment  is  unlikely  to  happen  accidentally.  Hopefully,  I propose  that  misaligned  instructional  systems 
(negative  T)  are  rare. 

The  Effect  of  Transfer  Alignment  Coefficient  on  Implementation.  The  combined  effect  of  transfer 
alignment  on  future  implementation  can  be  deduced  from  the  discussions  in  the  previous  sections.  A positive 
alignment  would  have  the  effect  of  increasing  implementation  because  teachers  would  be  more  likely  to  achieve  the 
appropriate  type  of  transfer  as  they  teach  with  the  instructional  system.  Teachers  would  be  willing  to  implement 
more  of  the  instructional  system  because  they  could  see  that  the  system  provided  an  appropriate  strategy  for  them  to 
teach  and  achieve  the  desired  results.  In  a way,  this  positive  transfer  alignment  would  have  the  effect  of  dampening 
the  frustration  of  high  levels  of  ECF.  A null  transfer  alignment  would  have  no  such  effect,  and  a negative  alignment 
would  actually  lower  implementation  of  even  instructional  systems  with  moderate  levels  of  ECF. 

Accumulated  Frustration  as  a function  of  increases  of  Implementation.  The  definition  of  the 
accumulated  frustration  variable  stated  that  when  teachers  suppress  their  creativity  in  order  to  increase 
implementation,  (7/+/  - IJ  < 0,  frustration  accumulates  proportionally  to  this  increase.  The  assumption  was  made 
that  when  implementation  decreases,  (lt+!  - IJ  < 0,  frustration  remains  static.  To  describe  this  relationship  between 
increases  of  implementation  and  accumulated  frustration,  Ah  the  following  Equation  8 is  proposed. 

Exportability  Control  Factor  as  a function  of  Accumulated  Frustration.  The  last  relationship  to  model 
is  the  one  between  ECF  and  At.  It  has  already  been  mentioned  that  feelings  of  frustration,  disappointment, 
dissatisfaction,  and  dejection  may  lead  to  second-order  change.  Below  a certain  threshold  value,  this  type  of  change 
is  unlikely.  Above  this  threshold,  change  is  imminent  (Dooley,  1997).  This  phenomenon  has  been  studied  in 
neuronal  patterns  (de  Bono,  1969)  and  managers’  decision-making  processes  (aspiration  theory;  Cyert  & March, 
1963;  Kiesler  & Sproull,  1982).  Likewise,  each  teacher  has  a tolerance  threshold  value  to  withstand  the  accumulated 
frustration.  Below  this  value,  teachers  deal  with  the  frustration  the  best  they  can,  slightly  adjusting  their  perception 
of  ECF,  due  to  the  familiarization  effect.  They  keep  trying  to  implement  the  instructional  system,  probably  changing 
the  implementation  mix  to  avoid  having  to  repress  their  creativity  so  much  the  next  time  they  use  the  system. 

If  the  instructional  system’s  ECF  is  not  very  high,  implementation  reaches  stability  before  the  frustration 
accumulates  beyond  the  tolerance  value.  However,  if  the  system’s  ECF  is  very  high  and  implementation  continues 
to  be  unstable,  more  frustration  accumulates  with  each  spike  in  implementation.  Eventually,  the  accumulated 
frustration  will  surpass  the  tolerance  threshold  value.  At  this  point,  what  can  teachers  do?  This  situation  leads 
teachers  to  drastically  change  the  instructional  system  to  a less  controlling  environment  by  changing  the 
instructional  system’s  ECF. 

What  are  some  of  the  behaviors  teachers  exhibit  that  so  drastically  change  their  perceptions  of  the 
instructional  system?  Under  the  right  conditions,  teachers  might  go  to  extreme  measures  in  order  to  create  a more 
stable  and  tolerable  environment.  Some  may  disregard  whole  parts  of  the  instructional  system;  others  may  quietly 
put  it  aside,  lobby  to  their  superiors  for  the  system’s  dismissal  or  change,  stop  reporting  to  their  superiors,  or  invent 
excuses  for  not  using  the  system— claiming  it  doesn’t  work  or  that  the  students  get  bored.  Some  teachers  may  even 
quit  their  jobs  in  order  to  escape  a controlling,  intolerable  situation;  such  cases  exemplify  the  ultimate  in  non- 
implementation. 
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Mathematically,  to  represent  the  second-order  change  relationship  between  ECF  and  Ah  with  slow  changes 
in  ECF,  below  the  threshold  value  and  a drastic  change  above  this  value,  Thom’s  cusp  catastrophe  equation  (Thom, 
1975;  Zeeman,  1976)  is  adapted,  as  shown  by  Equation  9. 

As  At  increases,  pushed  by  the  upward  shifts  in  implementation  (as  modeled  by  Equation  8),  Kt  diminishes. 
At  first,  Kt  diminishes  very  slowly,  then  as  At  approaches  the  fold  in  the  curve,  Kt  diminishes  at  an  increasing  rate. 
When  the  value  of  At  gets  to  be  higher  than  the  value  of  the  right  edge  of  the  fold,  Kt  drastically  jumps  to  the  lower 
part  of  the  curve.  This  sudden  change  in  ECF  represents  teachers’  decisions  to  respond  to  their  accumulated 
frustration  by  changing  the  instructional  system  and  transforming  it  into  a less  controlling  environment.  This 
behavior,  modeled  by  the  cusp,  is  called  a catastrophe  not  because  of  any  negative  connotation  of  the  word,  but 
because  a small  change  in  the  value  of  the  control  parameter  (At)  can  cause  a sudden  large  change  in  the  value  of  the 
independent  variable  ( Kt ). 

It  is  important  to  remember  that  the  catastrophic  change  only  occurs  if  frustration  accumulates  beyond  the 
teachers’  tolerance  threshold  level,  which  only  occurs  if  implementation  keeps  oscillating  in  an  unstable  fashion. 
This  happens  when  ECF  is  high  and  transfer  alignment  is  low,  or  both,  shows  the  DISE  model’s  behavior  when 
ECF  > 3 and  transfer  alignment  is  null. 

The  complete  DISE  model,  is  the  complete  graphical  representation  of  the  DISE  model.  Mathematically, 
the  complete  DISE  model  is  Equations  10,  11,  and  12  put  together  in  a system  of  three  equations,  shown  in  Equation 
13. 

The  DISE  model  of  implementation  for  teacher-mediated  instructional  systems  states  the  following:  (a)  The 
rate  of  implementation  is  a function  of  the  instructional  system’s  ECF  as  perceived  by  teachers,  the  previously 
experienced  rate  of  implementation  and  the  complementary  activities,  (b)  This  function  is  a balancing  loop  which 
leads  to  stable  rates  of  implementation,  (c)  A positive  transfer  alignment  coefficient  strengthens  the  perceived  ECF 
and  weakens  the  effect  of  complementary  activities;  in  general,  it  increases  rates  of  implementation,  (d)  If  ECF  is 
high  and  implementation  is  unstable,  teachers  accumulate  frustration  as  they  need  to  repress  their  creativity  when 
raising  the  rate  of  implementation,  (e)  Below  a tolerance  threshold  value  accumulated  frustration  reinforces  the 
perceived  ECF  due  to  familiarization  with  the  instructional  system,  (d)  If  implementation  remains  unstable  and 
frustration  accumulates  beyond  the  tolerance  point,  teachers  will  drastically  change  the  instructional  system  to  one 
with  a lower  ECF. 

Implementation  Scenarios 

The  DISE  model  of  implementation  for  teacher-mediated  instructional  systems  describes  five  distinct 
implementation  scenarios:  (a)  extinction,  (b)  low  stable,  (c)  high  stable,  (d)  unstable  with  second-order  change,  and 
(e)  aligned  high  implementations.  Each  scenario  is  discussed  next. 

Extinction  Implementation.  When  the  ECF  of  the  instructional  system  is  very  low  (between  0 and  1 when 
transfer  alignment  is  null,)  the  rate  of  implementation  falls  toward  0.  Left  to  themselves,  teachers  of  an  instructional 
system  with  such  a low  ECF  feel  disoriented  and  unsupported.  Teachers  cannot  find  enough  instructional  support  in 
the  system  to  implement  it,  so  they  eventually  abandon  it.  They  become  confused  and  uncertain,  and  at  the  same 
time  feel  pressured  to  perform  their  duties.  Under  these  conditions,  teachers  rapidly  abandon  the  new  system  and 
revert  back  to  their  old  teaching  habits,  or  improvise  and  create  their  own  solutions,  which  may  or  may  not  be  better 
than  the  new  system  but  which,  at  least,  provide  the  much-needed  equilibrium  the  teachers  are  seeking.  I have 
studied  several  cases  exemplifying  this  scenario,  for  example,  a case  where  teachers  in  an  elementary  school  attempt 
to  implement  but  quickly  abandon  a new  high-technology  classroom  program. 

This  extinction  can  be  correlated  with  Dooley’s  (1997)  third-order  change,  where  a particular  schema 
survives  or  dies  in  a Darwinian,  competitive  fashion.  In  other  words,  teachers  compare  what  they  are  expected  to  do 
against  what  they  have  been  doing,  and  decide  between  one  or  the  other.  In  this  case,  the  implementation  of  the 
instructional  system  dies  and  the  teachers’  complementary  activities  or  the  previous  “status  quo”  survive.  Notice  that 
this  extinction  occurs  no  matter  how  high  the  initial  rate  of  implementation  ( I0 ) was;  a high  I0  only  delays  the 
eventual  fate  of  the  implementation. 

Stable  Low  Implementation.  The  second  scenario,  when  the  ECF  is  between  1 and  2 (null  transfer 
alignment  coefficient),  can  be  called  Stable  Low  Implementation.  In  this  case,  the  instructional  system  provides 
enough  controls  to  help  teachers  try  the  system  on  their  own  and  appreciate  its  virtues.  However,  the  relatively  loose 
control  mechanisms  eventually  let  teachers  continue  using  their  current,  personal  instructional  methods  (fulfilling 
the  individual’s  need  for  creativity,  recognition,  and  security).  Teachers  adopt  only  those  features  of  the  system 
which  are  better  developed,  more  closely  match  their  instructional  philosophies,  and  better  fill  voids  in  their  current 
repertoire  of  instructional  materials.  Under  these  conditions,  the  DISE  model  predicts  that  teachers  will  consistently 
implement  only  a small  percentage  of  the  instructional  system  (less  than  half).  The  model  does  not  predict  which 
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parts  of  the  system  they  will  implement,  nor  whether  different  teachers  will  implement  the  same  parts.  A case  study 
essay  describing  a hypothetical  college  professor’s  implementation  of  a textbook  serves  as  example  for  this  scenario. 

Stable  High  Implementation.  When  the  ECF  of  the  instructional  system  is  moderately  high  (2  < ECF  < 3, 
T-  0),  the  system  reaches  the  Stable  High  Implementation  scenario  ( and  middle  right  part  in  Figure  1).  In  this  case, 
as  in  the  previous  one,  an  increase  in  the  system’s  ECF  achieves  an  increase  in  the  stable  rate  of  implementation. 
However,  gains  in  the  stable  rate  of  implementation  are  smaller  than  in  the  previous  case  (the  slope  of  the  curve  in 
Figure  1 diminishes  from  1 to  0 in  this  interval).  The  increased  control  included  in  the  instructional  system  affords 
teachers  a higher  degree  of  security,  guidance,  and  comfort  to  use  more  features  of  the  system.  However,  teachers 
still  feel  the  personal  craving  to  express  creativity,  so  they  take  up  a portion  of  the  available  contact  time  to 
implement  their  own  strategies,  discarding  what  they  consider  the  weakest  parts  of  the  instructional  system.  The 
DISE  model  predicts  that  for  systems  with  an  ECF  close  to  3,  teachers  would  implement  about  two-thirds  of  the 
original  instructional  system  in  a stable  fashion.  Again,  the  model  does  not  predict  what  parts  of  the  system  might  be 
implemented,  just  that  more  would  be  implemented  than  in  the  two  previous  scenarios. 

In  these  two  stable  scenarios,  during  the  first  few  iterations,  the  rate  of  implementation  oscillates  around  the 
converging  value.  These  oscillations  on  the  rate  of  implementation  can  be  explained  as  what  Argyris  and  Schon  term 
first-order  learning  (1978).  First-order  learning  occurs  when  agents  change  their  course  of  action  in  order  to  conform 
to  their  perceived  schema.  This  means  that  for  this  type  of  change  teachers  will  adjust  what  they  do  in  order  to 
conform  with  what  they  are  expected  to  do.  This  creates  a negative  feedback  loop  (like  the  balancing  loop  in  ) in 
which  actions  converge  toward  the  schema.  During  this  process  teachers  try  different  aspects  of  the  new 
instructional  system  until  their  actions  converge  with  the  new  schema.  At  the  end  of  this  process,  they  would  have 
achieved  a new  state  of  equilibrium.  This  part  of  the  implementation  process  is  what  the  DISE  model  attempts  to 
describe  by  showing  how  teachers  change  the  rate  of  implementation  (/,)  and  the  complementary  activities  (/,+/)  with 
each  iteration  until  they  reach  equilibrium. 

For  these  three  scenarios,  ECF,  changes  only  slightly,  if  at  all,  due  to  increased  familiarity  with  the 
instructional  system.  Equation  10  and  are  sufficient  to  model  these  three  scenarios. 

Unstable  Implementation  with  Catastrophic  Change.  The  next  scenario  corresponds  to  instructional 
systems  with  high  ECF  (3  < ECF  < 4,  T=  0).  It  can  be  called  Unstable  Implementation  with  Catastrophic  Change  (c 
and  right  side  of  Figure  1.)  An  instructional  system  with  ECF  this  high  imposes  so  many  expectations  on  teachers 
that  it  constrains  their  freedom.  Teachers  have  to  battle  an  internal  conflict  between  doing  what  they  are  told  to  do 
and  doing  what  they  want  to  do.  The  DISE  model  predicts  that  the  rate  of  implementation  jumps  back  and  forth 
between  high  and  low  values.  In  general  terms,  the  higher  the  ECF,  the  more  unpredictable  the  rate  of 
implementation  will  be.  This  unstable  implementation  does  not  allow  teachers  to  achieve  the  equilibrium  they  are 
seeking.  The  situation  becomes  untenable.  Teachers  in  this  situation  become  increasingly  frustrated  with  the 
instructional  system,  and  keep  trying  alternatively  to  implement  the  system  and  to  satisfy  their  creativity  until  they 
have  had  enough  and  decide  to  drastically  change  this  intolerable  situation.  Teachers  do  so  by  changing  the 
instructional  system  to  one  with  a much  lower  ECF.  Once  these  sudden  changes  occur,  implementation  converges  to 
the  appropriate  lower  rate  of  implementation. 

The  right  side  of  Figure  1 shows  7,’s  behavior  when  ECF,  > 3;  7,  does  not  settle  to  a single  value.  First,  7, 
suffers  a bifurcation  and  oscillates  between  two  values.  As  ECF,  increases,  7,  bifurcates  again  and  again  until  it 
reaches  a chaotic  region  in  which  7,  does  not  settle  but  oscillates  aperiodically  within  a certain  range  of  values.  As 
pointed  out  before,  c shows  the  time  series  graph  when  ECF,  = 3.8.  And  shows  the  unstable  oscillations  with  the 
catastrophic  change. 

In  my  research  I have  documented  two  case  studies  exemplifying  this  scenario.  One  of  them  shows  how 
tutors  changed  a highly  prescriptive  reading  program  to  a more  flexible  one.  The  other  one  relates  the 
implementation  of  a series  of  managerial  skills  training  modules  in  a business  environment. 

Aligned  High  Implementation.  The  previous  implementation  scenarios  seem  to  imply  that  stable  rates  of 
implementation  cannot  be  higher  than  two-thirds.  However,  these  cases  assumed  unaligned  instructional  systems  ( T 
= 0,  see  b).  As  has  been  discussed,  the  DISE  model  posits  that  positive  transfer  alignment  coefficients  increase 
implementation.  This  is  specially  true  for  instructional  systems  with  high  ECF.  Therefore,  a highly  aligned,  highly 
prescriptive  instructional  system  could  see  stable  rates  of  implementation  even  above  90  percent  (7,  = .90).  Teachers 
seem  to  be  more  willing  to  implement  a prescriptive  instructional  system  and,  if  needed,  even  to  repress  their 
creativity  when  the  alignment  between  learning  outcomes  and  instructional  strategies  is  evident.  This  is  depicted  in 
a. 

I have  encountered  two  case  studies  describing  the  aligned  high  implementation  scenario.  One  was  a highly 
prescriptive  two-day  training  workshop  with  very  specific  learning  outcomes  and  closely  aligned  instructional 
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strategies.  The  other  one  was  a highly  structured  reading  program  where  some  teachers  perceived  the  program’s 
positive  transfer  alignment  and  understood  its  design  and  were  able  to  maintain  implementation  at  high  rates. 

Generalized  Propositions  Derived  from  the  DISE  Model 

The  DISE  model  has  been  presented.  I have  discussed  the  model’s  scope,  assumptions,  variables, 
relationships,  and  scenarios.  This  last  section  discusses  those  generalized  propositions  I have  derived  from  the  DISE 
model.  It  is  important  to  remember  that  these  propositions  only  apply  to  instructional  systems  within  the  scope  of  the 
DISE  model:  teacher-mediated  instructional  systems.  These  propositions  are  offered  as  generalizations  derived  from 
the  model  and  suggest  guidelines  for  developers’  instructional  design  efforts.  Their  validity  is  subject  to  future 
theory  testing  and  theory  improvement  efforts. 

1.  The  rate  of  implementation  of  a teacher-mediated  instructional  system  is  dependent  on  the 
perceived  amount  of  exportability  controls  embedded  in  the  system  (ECF)  and  its  transfer  alignment  coefficient.  The 
effect  of  ECF  on  implementation  was  demonstrated  by  the  model  in  and  . The  effect  of  transfer  alignment  on 
implementation  was  depicted  in  . This  proposition  implies  that  instructional  developers  should  pay  purposeful 
attention  to  how  they  design  an  instructional  system’s  initial  training,  instructions,  support  resources,  schedules,  and 
accountability  methods  to  convey  the  desired  ECF  and  how  they  align  the  system’s  instructional  strategies  with  its 
learning  outcomes,  if  they  want  to  improve  their  chances  that  the  systems  gets  implemented  as  intended. 

2.  Normally,  the  transfer  alignment  coefficient  is  directly  related  to  the  rate  of  implementation.  This 
proposition  posits  that  it  is  unlikely  for  a misaligned  instructional  system  to  have  high  rates  of  implementation  (see  ). 
It  also  implies  that  instructional  systems  with  stable  high  rates  of  implementation  are  most  likely  positively  aligned. 

3.  The  final  rate  of  implementation  of  a teacher-mediated  instructional  system  is  not  dependent  on 
the  initial  rate  of  implementation.  This  was  clearly  shown  by  where  in  spite  of  different  initial  rates  of 
implementation,  the  final  stable  rate  of  implementation  did  not  change.  This  proposition  suggests  that  instructional 
developers  should  not  spend  undue  efforts  personally  emphasizing  high  initial  rates  of  implementation  while  they 
are  still  around,  and  implementation,  strictly  speaking,  has  not  yet  started.  Rather  they  should  concentrate  on 
communicating  the  correct  ECF  and  on  designing  highly  aligned  systems. 

4.  The  rate  of  implementation  changes  through  time  until  it  reaches  a state  of  equilibrium.  This 
proposition  was  modeled  by  the  outcome  of  the  different  scenarios  and  exemplified  with  case  studies.  According  to 
the  DISE  model,  even  for  very  different  conditions,  the  rate  of  implementation  changes  until  it  settles  into  a state  of 
equilibrium. 

5.  The  way  individual  teachers  perceive  a teacher-mediated  instructional  system ’s  ECF  and  transfer 
alignment  can  be  different  enough  to  greatly  affect  the  rates  of  implementation  over  time.  Teachers’  perceptions 
cannot  be  underestimated  nor  easily  dismissed.  Instructional  designers  should  attempt  to  estimate  these  perceptions 
beforehand  in  order  to  design  the  most  appropriate  ECF  and  transfer  alignment. 

6.  After  teachers’  implementation  reaches  a state  of  equilibrium,  only  significant  changes  to  these 
teachers ' perceived  ECF  and  transfer  alignment  could  change  the  rate  of  implementation.  Such  was  the  case  of  the 
implementation  I found  in  one  of  the  case  studies  where  a significant  change  in  the  way  the  master  trainers  taught 
the  certification  training  communicated  very  different  levels  of  ECF  to  the  teachers  and  they  changed 
implementation  accordingly. 

7.  A teacher-mediated  instructional  systems  with  ECF  below  a certain  level  (extinction  point) 
eventually  will  cease  to  be  implemented.  See  a.  It  is  important  to  point  out  that,  according  to  the  DISE  model,  this 
outcome  would  occur  no  matter  what  the  initial  rate  of  implementation  was.  Probably,  this  is  the  worst-case 
scenario,  because  it  wastes  the  most  resources.  Instructional  developers  should  be  careful  not  to  attempt 
implementing  instructional  systems  which  run  the  risk  of  suffering  extinction. 

8.  A teacher-mediated  instructional  system  with  ECF  above  a certain  level  (unstable  point)  will  be 
implemented  at  a unstable,  changing  rate  for  a period  of  time,  until  teachers  significantly  alter  the  instructional 
system  to  one  with  a lower  ECF.  I have  found  a few  examples  for  this  proposition  (see  ).  I consider  this  scenario  to 
be  the  next  worst  one  because  when  the  catastrophic  change  occurs,  all  bets  are  off  on  how  the  instructional  system 
will  be  implemented.  Implementation  could  fall  to  a low  stable  rate  or  even  below  the  extinction  point  to  zero. 

9.  Teacher-mediated  instructional  systems  with  moderate  ECF  (between  the  extinction  point  and  the 
unstable  point)  have  a non-zero,  stable  rate  of  implementation.  Several  cases  were  presented  to  illustrate  these 
scenarios  (see  and  b).  This  type  of  incomplete  or  partial  implementation  is  the  most  common  implementation 
outcome  for  teacher-mediated  instructional  systems.  This  conclusion  is  supported  by  Rogers’  (1995)  evidence  of 
how  prevalent  reinvention  of  innovations  is  and  by  the  effect  of  innovation  configurations  as  modeled  by  the  CBAM 
model  (Hall  & Loucks,  1981).  Other  studies  also  support  the  idea  that  teachers  adapt  the  instructional  system  and 
vice  versa  (Berman  & McLaughlin,  1974,  1976;  Davis,  Stand,  Alexander,  & Hussain,  1982;  Gephart,  1976). 
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1 0.  For  teacher-mediated  instructional  systems  with  moderate  ECF  (between  the  extinction  point  and 
the  unstable  point),  as  ECF  increases,  the  stable  rate  of  implementation  also  increases  but  it  does  so  at  a 
diminishing  rate.  This  is  shown  by  the  diminishing  slope  of  the  curve  in  Figure  1 and  . The  implication  of  this 
proposition  is  that  a developer’s  efforts  to  increase  implementation  by  increasing  the  instructional  system’s  ECF  will 
have  diminishing  returns  on  the  investment.  This  effect  could  prompt  the  developer  to  keep  increasing  the  level  of 
ECF,  to  achieve  higher  rates  of  implementation,  beyond  the  unstable  point.  This  could  generate  an  unfortunate 
outcome. 

1 1 . Initial  training  can  be  used  to  communicate  the  desired  ECF  and  transfer  alignment  to  teachers. 
Developers  should  pay  close  attention  to  this  important  component  of  ECF.  Nevertheless,  unless  they  are  planning 
to  personally  perform  this  initial  training  themselves,  if  developers  have  to  rely  on  master  trainers  to  deliver  the 
initial  training,  then  the  initial  training  itself  is  subject  to  the  implementation  dynamics  of  the  DISE  model. 

This  presentation  has  described  the  DISE  model  in  an  analytical,  step-by-step  fashion.  The  DISE  model 
presents  a theory  of  how  teacher-mediated  instructional  systems  might  be  implemented  over  time.  The  model’s 
scope  and  assumptions  were  carefully  delimited  and  identified.  The  variables  of  the  model  were  defined.  The  model 
was  then  represented  using  mathematical,  graphical,  and  narrative  representational  systems,  building  the  model’s 
relationships.  Five  distinct  implementation  scenarios  were  described.  Finally,  several  generalized  propositions 
derived  from  the  model  were  discussed,  suggesting  some  development  guidelines  for  instructional  developers. 
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Abstract 

One  of  the  problems  that  continues  to  plague  teachers  of foreign  languages  at  the  secondary  and  university 
level  is  the  students'  use  of  a “pony,"  i.e.,  a literal  translation , to  assist  them  in  completing  their  translation 
assignments.  Due  to  the  wide  availability  of  such  translations  and  the  tendency  for  students  to  choose  the  path  of 
least  resistance,  the  use  of  such  crutches  is  not  bound  to  end.  Though  I would  not  suggest  that  it  is  appropriate  in 
all  instances,  it  is  my  contention  that  as  teachers  of foreign  languages,  we  can  guide  the  students'  use  of  the  “pony" 
in  ways  that  will  work  to  their  advantage  rather  than  their  detriment,  as  can  be  demonstrated  through  a popular 
language  textbook  from  the  early  17th  century.  The  example  discussed  in  this  paper  and  an  examination  of  the 
design  methodology  used  by  the  author  of  the  17th  century  textbook  also  suggest  the  encouraging  conclusion  that  a 
new  historical  approach  to  instructional  design  is  appropriate  and  beneficial  for  modern  instructional  designers, 
especially  vis-a-vis  modern  foreign  language  pedagogy. 

Introduction 

In  this  age  of  technological  innovation  and  “advances,”  and  especially  in  this  very  modem  field  of 
instructional  design,  a new  historical  approach  to  pedagogy  might  seem  oddly  misplaced.  However,  since  we  are 
confronting  issues  that  are  literally  millennia  old  and  that  have  been  debated  in  the  laboratories  of  experience  for 
many  centuries,  we  should  not  dismiss  out  of  hand  the  solutions  proposed  and  tested  by  teachers  throughout  the 
ages. 

As  both  Classicist  and  instructional  designer,  my  question  is  how  a new  historical  approach  to  Classical 
language  pedagogy,  when  viewed  through  the  lenses  of  modem  instmctional  design,  informs  foreign  language 
pedagogy  today.  Though  my  interests  and  examples  deal  primarily  with  teaching  ancient  Greek  and  Latin,  the 
methods  revisited  in  this  paper  can  be  easily  applied  to  teaching  any  non-native  language. 

At  the  forefront  of  the  controversy  in  Classical  language  pedagogy  in  the  last  ten  years  are  issues  of  the 
changing  student  profile.  The  complaints  in  the  pedagogical  discourse  form  a remarkably  consistent  refrain:  the 
students  will  not  study,  the  students  do  not  have  a foundational  knowledge  in  grammar,  the  students  have  no 
motivation,  the  students  have  other  priorities  (Abbott,  1991;  Gruber-Miller,  1998;  Kitchell  et  al.,  1996;  Phinney, 
1989;  Sebasta,  1998). 

This  issue  of  the  new  model  of  student  has  also  been  at  the  forefront  of  instructional  design  and  educational 
theory  in  recent  years,  but  has  been  cast  in  a much  more  objective  light.  Rather  than  making  a value  judgment  about 
their  essential  quality,  intelligence,  or  background  knowledge,  educational  researchers  and  theorists  have  attempted 
to  describe  how  students  learn;  instructional  designers,  building  on  the  findings  of  these  educational  researchers, 
have  classified  types  of  knowledge  and  produced  theories  describing  how  students  learn  each  type  best. 

In  recent  years,  in  an  attempt  to  shake  off  the  shackles  of  the  traditional  “grammar-translation”  method  of 
language  pedagogy  inherited  from  their  teachers  and  their  teachers  before  them,  Classical  language  teachers  have 
largely  switched  over  to  a “more  intuitive”  and  more  modem  approach  dubbed  the  “reading”  method  (Bums  & 
O’Connor,  1987;  Davis,  1991;  Gruber-Miller,  1998;  Knudsvig  & Ross,  1998;  Phinney,  1989;  Sebasta,  1998).  The 
reading  method,  though  a step  in  the  right  direction,  is  not,  however,  a panacea.  We  must  continue  to  look  in  all 
comers,  however  unlikely,  for  new — or  old — ideas  that  will  help  inspire  language  pedagogy  with  new  life.  It  is  in 
this  spirit  that  I suggest  it  is  time  the  past  was  exhumed  and  examined  in  the  new  light  provided  by  modem  theories. 
The  eminent  Hellenistic  historian  Peter  Green  noted  that  historical  interpretation  is  largely  affected  by  the  Zeitgeist 
of  the  era  in  which  the  historian  works  (1993).  Thus,  issues  considered  anew  in  each  succeeding  generation 
continue  to  yield  rich  insights.  Unfortunately,  the  historical  approach  to  Classical  language  pedagogy  has  been 
largely  dispensed  with  in  favor  of  more  “modem”  approaches  (Phinney,  1989).  Proponents  of  these  approaches 
claim  that  they  approximate  a more  authentic  learning  experience.  This  goal  is  a worthy  one,  and  the  efforts  that 
have  been  made  should  be  commended.  However,  by  analogy  with  the  historical  insights  gained  in  the  last 
generation  in  areas  such  as  Hellenistic  history,  gender  studies,  and  oral  poetic  composition,  I contend  that  the 
distantly  historical  approaches  to  Classical  language  pedagogy  can  reveal  unexpected  treasures  when  reviewed  using 
a similar  new  historical  approach. 
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For  the  remainder  of  this  paper  I will  focus  on  a few  of  the  advantages  that  could  accrue  from  employing 
one  example  of  these  “old”  techniques  and  methodologies  to  the  modem  foreign  language  classroom.  Many  of 
these  techniques  do  not  simply  provide  a way  to  work  around  “obstacles”  presented  by  the  new  profile  of  student 
characteristics;  they  actually  work  with  the  new  generation  of  students  and  are,  in  addition,  validated  for  modem  use 
by  modem  educational  and  instructional  design  theories. 

Problems  of  Foreign  Language  Pedagogy 

Perhaps  the  most  daunting  problem  facing  foreign  language  students  today  is  the  tremendous  amount  of 
cognitive  resources  required  for  translation  (what  instructional  designers  would  call  “authentic  whole-task 
practice”).  While  translating  a single  sentence  students  must  analyze  parts  of  speech,  morphology,  denotation  of 
words,  connotation  of  words,  grammatical  syntax,  and  word  order,  and  then  reconstruct  the  pieces  into  an  idiomatic 
rendering  in  their  native  language;  add  to  this  the  pressure  of  “performing”  in  front  of  peers  and  instructor  or  for  a 
grade,  and  it  becomes  apparent  that  even  in  its  simplest  form  (i.e.,  for  the  shortest  sentences),  the  act  of  translation 
produces  a nearly  constant  state  of  cognitive  overload. 

The  teacher’s  most  common  solution  to  the  problem  of  cognitive  overload  is  to  deconstruct  the  practice  as 
far  as  possible  into  its  various  component  parts,  as  is  evidenced  by  the  plethora  of  vocabulary  drills,  memorization 
of  grammar  rules,  and  morphology  drills  available  in  both  computerized  and  paper  form  (Latousek,  1998). 
However,  the  connotation  of  vocabulary  can  only  be  understood  in  context,  that  is,  in  conjunction  with  other  words, 
and  therefore  cannot  be  effectively  practiced  simply  by  drilling  single  word  equivalents  as  though  they  were  the 
times  tables.  The  same  is  true  for  appreciating  the  style  and  interpreting  the  tone  of  an  author  or  passage.  The  result 
of  drilling  individual  component  skills  is,  consequently,  a decontextualized,  fragmentary  type  of  knowledge  that  is 
often  difficult  to  integrate  during  authentic  whole-task  practice,  i.e.,  translation.  The  dilemma,  though,  is  that 
without  making  some  of  these  component  skills  recurrent  or  automatic  by  drill-type  practice,  the  task  of  translation 
is  essentially  hopeless  for  the  novice. 

The  obvious  solution  to  the  instructional  designer  is  to  scaffold  the  authentic  task  at  various  levels  by 
recombining  component  skills  to  form  intermediate  practice  tasks.  But  identifying  a solution  is  the  easy  part.  The 
more  difficult  question  is  how  to  implement  that  solution.  It  is  for  answers  to  difficult  problems  like  this  that  we  can 
and  should  look  to  the  past  for  illumination. 

Bathe’s  Language  Methodology 

In  the  early  17th  century,  William  Bathe,  an  Irish  Jesuit  stationed  at  Salamanca,  Spain,  wrote  a stunningly 
successful  language  manual,  the  Janua  Linguarum  [The  Gate  to  Languages ],  based  on  his  innovative  pedagogical 
theory.  The  textbook,  designed  for  students  of  both  modem  and  “scholarly”  languages,  saw  numerous  editions 
published  throughout  the  17th  century  and  was  used  to  teach  at  least  8 different  languages.  Surprisingly,  1 1,000 
copies  were  printed  in  Puritan  England  alone — though,  for  obvious  political  reasons,  without  attribution  to  its  Jesuit 
author.  Bathe’s  book  also  formed  the  basis  for  several  other  pedagogical  methods.  For  example,  upon  comparison 
with  Bathe’s  text,  the  Janua  Linguarum  Reserata  of  John  Comenius,  which  has  come  to  be  regarded  as  the  first 
modem  “textbook”  in  the  West,  is  discovered  to  be  little  more  than  a cheap  imitation  of  this  well-known  and  widely- 
acclaimed  text  (Corcoran,  1911). 

As  an  instructional  designer,  who  by  definition  believes  that  a systematic  approach  to  designing  instruction 
will  produce  a superior  instructional  product,  I consider  the  overwhelming  success  of  the  17th  century  textbook  a 
natural  result  of  an  underlying  systematic  instructional  design.  The  introductory  matter  reveals  an  astounding 
anticipation  of  ISD  models  as  well  as  a deep  understanding  and  appreciation  of  factors  that  significantly  affect 
learning,  as  the  following  discussion  and  passages  from  Bathe’s  text  will  show. 

Analysis 

As  Allison  Rossett  so  convincingly  demonstrates  in  her  recent  exegesis  (1999),  the  analysis  phase  of  the 
design  process  is  essential  to  the  success  of  the  design  itself.  Bathe  was  not  unaware  of  the  importance  of  defining 
the  need  for  his  method,  the  target  population  for  his  manual,  the  component  tasks  that  constitute  the  authentic 
whole  task  (i.e.,  translation),  and  the  objectives  on  which  he  would  focus  in  creating  his  manual  and  explicating  his 
method. 

Needs  Analysis 

In  the  second  chapter  of  the  introduction  to  his  first  edition  of  the  Janua , Bathe  identifies  the  need  for  his 
new  language  methodology.  He  states: 
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For  learning  languages.. .only  two  ways  have  yet  been  found:  the  ‘rule-way,’  such  as  grammar 
applied  to  note  the  concord  of  words,  and  the  ‘ruleless-way,’  the  ordinary  plan  of  learners  of 
common  tongues  through  reading  and  speaking.  The  two  ways  are  related  to  each  other,  in  that 
the  ‘rule-way’  gives  more  sureness,  where  the  ‘ruleless-way’  more  ease  in  learning.  The  former  is 
to  be  preferred  where  the  language  is  not  in  common  use,  the  latter  in  case  of  a [modem]  tongue. 

But  if  a ‘via  media’  [middle  way]  can  be  thought  out,  which  would  equal  the  ‘rule-way’  in 
sureness,  and  the  ‘rule less- way’  in  ease,  it  would  beyond  question  be  placed  several  degrees 
higher  than  either.  Such  a way  we  have  here.. .undertaken  to  point  out.  (as  cited  in  Corcoran, 

191 1,  p.  72). 

It  is  interesting  that  several  centuries  before  Dave  Merrill  arrived  on  the  instructional  design  scene,  Bathe  was  driven 
by  the  mandate  to  make  instruction  more  effective  and  efficient  and  consequently  more  appealing. 

Target  Population  Analysis 

Bathe  defines  the  target  audience  for  his  method  in  the  preface  to  the  first  edition  of  the  Janua  Linguarum : 
“this  manual,”  he  says,  “is  chiefly  for  those  men  who  are  missionaries  in  foreign  regions... to  leam  the  foreign 
languages”  in  use  where  they  were  serving.  “It  is  also  for  confessors,”  he  continues,  “so  that  they  can  understand 
the  meaning  of  the  feelings  concealed  in  the  hearts  of  the  foreign  peoples...,  especially  for  confessors  in  those  areas 
which  foreigners  frequent.”  He  adds  that  the  book  is  for  “those  of  advanced  years  who  have  been  deterred  from 
taking  the  orders  by  the  tediousness  of  grammatical  studies.”  It  is  also,  he  says,  for  those  “for  whom  the  work  of 
learning  vocabulary  for  many  years  is  vexing,  who  do  not  wish  to  undertake  the  work  which  is  common  to  the 
‘intricate’  method  in  popular  use.  This  manual,”  he  says,  “yields  a more  fruitful  command  of  vocabulary  for 
students  of  grammar  and  rhetoric  in  three  months  than  any  other  will  in  three  whole  years;  this  is  true,”  he  claims, 
“not  only  of  the  vocabulary  of  the  modem  languages,  but  also  for  Hebrew,  Greek,  and  Latin.”  He  goes  on  to  say: 

It  is  very  appropriate  for  teachers  who  wish  to  teach  the  fundamentals  of  the  vocabulary  that 
occurs  in  all  the  authors,  for  many  words  in  the  works  examined  in  the  trivium  courses  are 
contained  in  this  volume.  And  it  will  also  be  helpful  for  travelers  who  will  by  using  it  be  able  in 
the  shortest  time  possible  to  collect  a ‘forest’  of  words  used  in  another  country.  It  will  also  useful 
for  those  who  are  involved  in  various  business  dealings,  for  example,  those  who  are  sent  to  speak 
with  the  heads  of  state,  for  learning  foreign  idiomatic  usages  in  a short  time.  And  it  will  prove 
useful  for  remedying  the  negative  effects  of  not  being  able  to  attend  a public  school  for  servants  in 
noble  houses.  It  is  for  those  who  wish  to  spare  the  expense  of  years  of  study  in  the  humanities. 

And  it  will  assist  those  who  wish  to  leam  the  noble  modem  languages,  such  as  Italian,  Spanish, 

German,  and  French,  in  the  comprehension  of  vocabulary,  (as  cited  in  Corcoran,  1911,  pp.  266- 
269). 

Task/Objectives  Analysis 

In  the  first  chapter  of  his  introduction,  Bathe  divides  the  learning  of  languages  into  four  elements  or 
component  skill  areas:  words,  syntax,  idiomatic  phrases,  and  style  (Corcoran,  1911).  This  four-fold  division 
resembles  a modem  high-order  task  analysis.  Then,  in  chapter  three,  he  suggests  that  while  both  vocabulary  and 
syntax  could  effectively  be  learned  using  his  “via  media,”  the  first  edition  of  his  text  would  deal  only  with  the  first 
element.  From  his  discussion  throughout  the  introduction,  it  can  be  conjectured  that  Bathe  intended  to  add  to  the 
text — or  perhaps  even  create  an  additional  manual — to  explicate  his  method  for  learning  syntax  using  this  “middle 
way”  method.  However,  he  died  shortly  before  the  publication  of  his  first  edition  and  was  unable  to  accomplish  his 
plan;  but  two  translators/editors  of  later  editions  of  Bathe’s  work  took  his  plans  to  heart  and  found  a way  to  teach 
syntax  using  his  “via  media.”  These  two  editions  will  be  discussed  later.  For  now,  let  us  remark  on  the  practical 
wisdom  Bathe  demonstrated  by  focusing  on  an  objective  relating  to  a single  element  in  his  task  analysis  for  the 
prototype  of  his  method. 

Design 

Bathe’s  prescient  anticipation  of  modem  instructional  design  methodology  evident  in  his  extensive 
analyses  of  need,  audience,  tasks,  and  objectives  extended  to  the  design  phase,  as  well.  As  with  the  analysis,  the 
primary  evidence  for  his  attentiveness  can  be  found  in  the  introductory  matter  of  the  first  edition.  Chapters  three 
through  seven,  nine,  and  ten  make  explicit  the  plan  behind  the  method  and  give  guidelines  for  implementing  it  in 
later  editions.  Chapter  eight,  in  fact,  bears  a startling  resemblance  to  a management  plan.  Bathe  leaves  explicit 
instructions  for  those  who  will  perpetuate  his  work  about  how  to  do  so  in  accordance  with  his  original  design 
(Corcoran,  1911). 


374 


381 


Because  Bathe  passed  away  before  his  product  could  be  implemented,  evaluated,  and  revised,  that  part  of 
the  ISD  process  were  of  necessity  left  to  others.  As  mentioned  above,  two  of  the  editors  of  later  editions  of  the 
Janua  added  to  the  manual  exercises  to  assist  the  learner  in  assimilating  the  second  element  of  language  identified 
by  Bathe:  the  syntax.  There  are  anecdotal  records  of  experiments  with  the  manual  to  gauge  its  effectiveness 
(Corcoran,  191 1);  the  results  of  these  experiments  are  commonly  reported  as  testimonials  in  the  prefatory  material  of 
various  editions  of  the  textbook  and  are  therefore  rather  historically  suspect.  However,  the  important  point  is  that 
some  sort  of  evaluation  was  occurring  per  Bathe’s  instructions. 

Bathe’s  Ideas  about  Learning 

As  an  educator,  Bathe  was  aware  of  many  of  the  problems  currently  being  discussed  by  modem  educational 
theorists.  For  example,  in  chapter  three  of  the  introduction  to  the  Janua , Bathe  explicitly  acknowledges  the 
problems  of  cognitive  load  and  the  decontextualization  of  knowledge  caused  by  rote  memorization  of  definitions 
and  rules.  “How  is  it,”  he  asks,  “that  in  learning  syntax  some  adopt  the  ‘rule-way,’  and  some  the  ‘rule less-way,’ 
while  in  learning  words  no  one  sets  before  himself  the  ‘rule-way,’  by  thoroughly  learning  a series  of  words?  For 
this,  three  reasons  can  be  given.  First,  that  vocabularies  contain  many  unusual  words,  useless  for  the  purposes  of 
many  learners.  Secondly,”  he  continues,  “that  a close  connection  exists  between  many  words,  and  so,  when  one 
fundamental  word  is  known  as  the  source  of  others,  these  are  very  easily  inferred  from  it....  When  the  meaning  of 
one  of  them  is  known  beforehand,  the  rest  easily  follow  and  so  do  not  call  for  any  special  effort  on  the  part  of  a 
learner.”  And  finally,  he  adds,  “the  third  and  main  reason  is  that  words  in  a vocabulary  [list]  lack  significance;  from 
this  it  follows  that  the  memory,  deprived  of  the  assistance  of  the  intelligence,  cannot  keep  firm  hold  on  them”  (as 
cited  in  Corcoran,  1911,  pp.  72-73).  Bathe’s  arguments  bear  an  unmistakable  resemblance  to  the  tenets  of 
Meaningful  Reception  Learning  as  explicated  by  Ausubel,  Novak,  and  Hanesian  (1978). 

Bathe’s  perceptiveness  when  it  comes  to  learning  and  to  the  design  of  instructional  materials,  especially 
when  his  work  is  compared  to  modem  theories  and  principles,  was  phenomenal.  Not  only  does  his  acuity  lend 
credence  to  his  historically-acclaimed  position  as  one  of  the  preeminent  educators  in  the  post-Renaissance  West 
(Nolte,  1768,  as  cited  in  Corcoran,  1911),  but  it  also  suggests  that  modem  foreign  language  teachers  and 
instructional  designers  alike  could  derive  great  benefits  from  examining  and  applying  certain  elements  of  his 
methodology  and  techniques.  The  following  section  of  this  paper  will  provide  an  example  of  how  one  such 
technique  found  in  this  historical  textbook  could  be  applied  to  modem  foreign  language  pedagogy. 

A New  Historical  Approach  to  an  Old  Problem 

One  of  the  problems  that  continues  to  plague  teachers  of  foreign  languages  at  the  secondary  and  university 
level  is  the  students’  use  of  a “pony,”  i.e.,  a literal  translation,  to  assist  them  in  completing  their  translation 
assignments.  Due  to  the  wide  availability  of  such  translations  and  the  tendency  for  students  to  choose  the  path  of 
least  resistance  when  it  comes  to  completing  homework,  the  use  of  such  crutches  is  not  bound  to  end.  Though  I 
would  not  suggest  that  it  is  appropriate  in  all  instances,  it  is  my  contention  that  as  teachers  of  foreign  languages,  we 
can  use  supplemental  exercises  to  guide  the  students’  use  of  the  “pony”  in  ways  that  will  work  to  their  advantage 
rather  than  their  detriment,  as  can  be  demonstrated  through  two  specific  examples  from  Bathe’s  textbook. 

The  Arrangement  of  Bathe’s  Book 

One  of  the  more  interesting  aspects  of  Bathe’s  text — and  one  of  the  most  informative  for  the  problem  of 
modem  foreign  language  pedagogy  posed  above — is  that  it  employed  facing  text  and  translation,  much  as  a modem 
“pony”  does  today.  The  sentences  in  the  language  to  be  acquired  were  found  on  the  right-hand  leaf  and  the  literal 
translation  of  the  sentences  into  the  native  language  on  the  left.  Some  editors  made  the  translations  interlinear  while 
others  made  multiple  columns  in  order  to  show  the  sentences  rendered  into  as  many  as  six  languages  across  the 
double  leaf  (Corcoran,  1911). 

Though  the  arrangement  of  the  book  appears  at  first  glance  to  be  more  an  issue  of  message  design  than  of 
instructional  design,  the  exercises  that  encourage  the  learners  to  actively  engage  both  the  text  and  translation 
simultaneously  and  to  manipulate  the  material  in  a way  that  facilitates  the  abstraction  of  vocabulary  and  principles 
of  syntax  reveal  the  centrality  of  message  design  in  the  overall  instructional  design. 


Harmar’s  English  Edition 

John  Harmar,  Regius  Professor  of  Greek  at  Magdalen  College,  Oxford,  edited  and  oversaw  publication  of 
two  English  editions  of  the  Janua  Linguarum.  Harmar,  in  the  introduction  to  his  first  edition  (the  sixth  English 
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edition,  published  in  1623),  reports  in  detail  the  alterations  he  saw  fit  to  make  to  the  Janua , alterations  that  were 
“sufficiently  commended  both  by  diverse  experiments,  and  by  five  editions  past.”  These  changes  included  making 
the  translation  “more  significant  and  correspondent  to  the  Latin,  and  more  clearly  to  reflect  on  it”  (as  cited  in 
Corcoran,  1911,  p.  98).  That  is,  he  made  the  vernacular  translation  more  precisely  literal.  He  also  refined  the 
Classical  Latinity  of  the  sentences  themselves  so  they  would  be  of  more  immediate  use  to  teachers  and  students  of 
Classical  Latin.  Finally,  he  added  directions  for  manipulating  the  Latin  text  in  such  a way  as  to  make  the  “concord 
of  words”  (i.e.,  syntax  and  word  order),  the  second  element  of  language  identified  by  Bathe,  explicit  to  the  students: 
he  directs  the  students  “to  construe  by  letters  pointing  out  the  grammatical  position  and  sequence  of  the  words”  (as 
cited  in  Corcoran,  p.  99).  The  student,  when  faced  with  a sentence  such  as  the  one  that  follows  (sentence  number  9 
of  1200  in  the  Janua),  Hoc  momentum,  unde  pendet  aeternitas , would  have  written  something  approximating  the 
following  string  of  letters:  Adj[ective]  N[oun]  Adv[erb]  V[erb]  N[oun],  thereby  indicating  the  part  of  speech  of  each 
word  in  the  order  in  which  it  appears  in  the  language  to  be  acquired. 

Harmar  apparently  believed  that  with  the  modifications  described  above  the  book  would  displace  all  other 
beginning  Latin  texts,  which  he  styles  “elementary  trash,”  for  teaching  the  fundamentals  of  Classical  Latin 
(Corcoran,  1911).  Indeed,  by  increasing  the  Latinity  of  the  sentences  and  the  correctness  of  the  translations,  he 
prepared  a text  suitable  for  teaching  all  four  of  Bathe’s  elements:  vocabulary,  syntax,  idiomatic  phrases,  and  style. 
For  as  the  students  actively  manipulated  the  text,  they  would  of  necessity  have  encountered  phrases  and  idiomatic 
translations  as  well  as  the  general  style  of  the  Latin  language. 

Applications  to  Modern  Foreign  Language  Pedagogy 

Harmar’s  pedagogical  method  could  be  easily  adapted  to  modem  use  for  both  ancient  and  modem 
languages.  Working  with  a facing  literal  translation,  students  could  be  assigned  to  use  letters  to  indicate  the  parts  of 
speech  of  each  element  in  a sentence  in  a passage  of  text  by  the  author  being  studied.  To  increase  the  difficulty  of 
the  exercise,  the  students  could  be  instructed  to  indicate  the  syntactical  function  of  each  word  within  the  sentence  as 
well  as  its  part  of  speech  and  to  note  differences  in  syntactical  function  or  idiom  between  the  two  languages. 
Ideally,  the  passage  used  for  this  exercise  would  be  different  than  the  passage  used  for  whole-task  practice  but  by 
the  same  author. 

Several  instructional  advantages  could  be  derived  from  such  an  exercise.  First,  the  students  would  be  able 
to  focus  on  syntax  and  word  order  without  any  but  the  most  cursory  attention  to  the  semantics  of  the  sentence;  such 
would  be  the  advantage  of  having  a literal  translation  readily  available  for  consultation.  This  type  of  scaffolding 
reduces  the  cognitive  load,  thereby  increasing  the  cognitive  resources  available  to  attend  to  the  syntactical  patterns. 
Second,  with  sufficient  practice,  the  students  will  eventually  assimilate  schemata  for  the  syntactical  patterns  not  only 
of  the  language  in  general  but  also  of  the  specific  author  under  consideration.  Each  additional  sentence  analyzed  in 
this  way  will  assist  in  the  tuning  of  these  syntactical  schemata  (van  Merrienboer,  1997).  Third,  as  Bathe  himself 
pointed  out  with  regards  to  learning  vocabulary,  the  syntax,  when  situated  in  an  authentic  environment  is  much 
easier  to  learn  and  remember  than  when  learned  as  a list  of  decontextualized  rules.  This  last  point,  again,  is  the 
basic  tenet  of  the  Meaningful  Reception  Learning  theory  (Ausubel  et  al.,  1978). 

The  Portuguese  Edition 

In  1623,  Dom  Mauro  de  Roboredo  published  his  Porta  de  Linguas , a Portuguese  translation  of  Bathe’s 
Janua.  He  expresses  in  the  preface  his  intent  to  “extend  the  usefulness  of  the  original  edition  by  adding,  on  its  own 
expressed  plan,  a Portuguese  version”  (as  cited  in  Corcoran,  1911,  p.  107).  In  addition  to  simply  adding  a literal 
Portuguese  translation  of  the  Latin  sentences,  he  made  slight  changes  to  the  message  design,  as  well.  Roboredo 
retained  both  the  Latin  and  Spanish  sentences  in  their  original  positions  but  added  his  Portuguese  translation  above 
the  Spanish  translation  on  the  left-hand  side  of  the  page.  Then  he  connected  the  corresponding  elements  of  the 
sentences  in  all  three  languages  with  numerals  (Corcoran).  Using  the  same  sentence  from  the  Janua  as  in  the 
previous  example,  an  example  of  Roboredo’s  rendering  would  have  been  as  follows:  Hoc  momentum , unde  pendet 
aeternitas.  A literal  English  translation  following  the  same  plan  would  read:  This  is  the  moment  from  which 
eternity  hangs. 

Applications  to  Modern  Foreign  Language  Pedagogy 

This  unique  element  of  message  design  found  in  the  Portuguese  edition  of  the  Janua  suggests  possibilities 
for  use  in  modem  foreign  language  instruction.  Using  a passage  of  text  in  a facing  literal  translation,  the  students 
could  be  assigned  to  use  numerals  to  connect  corresponding  elements  in  the  sentences  of  the  foreign  and  native 
languages. 
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The  advantages  to  this  exercise  would  be  several.  First,  the  exercise  would  encourage  students  to  actively 
engage  both  text  and  translation  simultaneously,  thereby  drawing  attention  to  both  the  foreign  word  and  its 
equivalent  in  the  students’  native  tongue  and  taking  maximum  advantage  of  the  scaffolding  inherent  in  such  a 
format.  Second,  the  students  would  be  exposed  to  much  more  vocabulary  than  they  would  be  if  they  were  expected 
to  look  up  each  unfamiliar  word  in  a dictionary  (van  Merrienboer,  1997).  Furthermore,  the  students  would  have  at 
their  fingertips  a model  of  expert  translation:  they  would  see  the  various  nuances  of  a particular  word  and  also  how 
the  literal  definitions  of  words  are  influenced  by  surrounding  words  to  form  a coherent  thought  or  idiomatic 
expression  (van  Merrienboer).  Fourth,  students  are  more  likely  to  remember  vocabulary  when  they  encounter  new 
vocabulary  words  in  the  context  of  a complete  thought,  as  Bathe  himself  pointed  out.  Today  we  would  muster  the 
tenets  of  MRL  as  support  for  this  statement  (Ausubel  et  al.,  1978).  Fifth,  the  second,  third,  and  fourth  elements  of 
language,  as  articulated  by  Bathe  (i.e.,  syntax,  idiomatic  phrases,  and  style)  would  also  be  called  to  the  students* 
attention  as  they  moved  systematically  through  the  passage;  and,  as  mentioned  earlier,  because  the  semantics  would 
not  be  tying  up  their  cognitive  resources,  a greater  amount  of  attentional  resources  would  be  free  to  assimilate 
syntactical  schemata,  idiomatic  renderings  of  certain  phrases,  and  the  stylistic  patterns  of  the  author  under 
consideration  (van  Merrienboer). 

Conclusions 

Contemporary  testimonials  and  the  overwhelming  popularity  of  the  book  on  the  basis  of  numbers  alone 
aver  that  the  exercises  contained  in  the  Janua  Linguarum  were  successful  for  teaching  the  rudiments  of  vocabulary 
and  syntax  in  the  17th  century;  but  their  effectiveness  then  and  their  potential  for  effectiveness  now  can  also  be 
verified  by  their  correspondence  to  modem  instructional  design  principles  and  learning  theories.  The  major 
principles  exemplified  in  the  Janua  Linguarum  include:  1)  scaffolding  novices  in  several  component  skill  areas  to 
allow  them  to  successfully  engage  in  authentic  practice  activities  as  soon  as  possible  (van  Merrienboer,  1997);  2) 
reducing  cognitive  load  by  reducing  the  number  of  component  skills  being  practiced  at  one  time  (van  Merrienboer); 
3)  modeling  expert  translation  (van  Merrienboer);  4)  increasing  the  automaticity  of  rule-based  or  recurrent 
component  skills  through  more  encounters  with  vocabulary  and  syntax  (van  Merrienboer);  5)  assisting  the 
assimilation  and  tuning  of  syntactical  schemata  by  encouraging  students  to  make  patterns  explicit  through  the  use  of 
semantically  insignificant  symbols  (van  Merrienboer);  6)  requiring  thoughtful  engagement  from  the  learner  (van 
Merrienboer);  7)  encouraging  learners  to  assimilate  vocabulary  from  a meaningful  context  rather  than  by  rote 
memorization  (Ausubel  et  al.,  1978);  and  8)  increasing  students’  motivation  to  leam  by  providing  authentic  early 
experiences  with  the  texts  (Keller,  1987). 

It  is  my  argument  that  the  adherence  of  the  exercises  from  the  Janua  Linguarum  discussed  in  this  paper  to 
principles  of  good  instructional  design  justify  not  only  reinstating  this  antique  use  of  the  “pony”  in  modem  foreign 
language  pedagogical  methods,  but  also  justifies  using  a new  historical  approach  to  instructional  design  to  infuse  our 
modem  approaches  to  foreign  language  pedagogy  with  the  sustaining  lifeblood  of  tried  and  true  methodology. 
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Audience  Objectives 

1.  AECT  audience  members  will  be  able  to  describe  and  evaluate  the  results  of  a school-university  collaboration 
that  is  part  of  a web-based  high  school  project. 

2.  AECT  members  will  be  able  to  briefly  describe  the  development  and  evaluation  of  four  components 
developed  as  part  of  the  project:  1.  a teacher-training  and  support  needs  assessment,  2.  a "rookie  camp"  to 
support  students  in  using  WebCT,  3.  an  instructional  unit  to  help  teachers  learn  about  digital  copyright,  and  4. 
a web-based  "courselet"  to  teach  students  about  key  elements  of  the  Odyssey  of  Homer. 

Background 

One  high-school  district  in  the  Phoenix,  Arizona,  metropolitan  area,  has  over  the  past  six  years,  built  its 
infrastructure  to  support  high-quality  technology  integration  by  its  teachers.  At  one  high  school,  McClintock  High  in 
Tempe,  the  technology  coordinator,  administrators,  and  several  innovative  teachers  determined  that  the  time  had  come  to 
take  advantage  of  what  technology  could  now  do  for  their  teachers  and  students.  They  knew  their  school  was  ready  to 
take  the  next  step  toward  true  technology  integration. 

The  school  personnel  saw  this  next  step  consisting  of  developing  several  levels  of  web-based  and  web- 
supplemented  instructional  materials  and  programs.  At  one  level,  they  partnered  with  the  Virtual  High  School  (VHS) 
out  of  Concord,  Massachusetts,  to  secure  access  to  advanced  placement  courses  for  students  who  could  not  otherwise 
take  these  courses.  One  teacher  enrolled  in  the  VHS's  teacher-training  program.  During  this  year,  five  McClintock 
students  were  able  to  take  such  courses  as  Advanced  International  Politics  and  Advanced  Microbiology,  all  fully  web- 
based  courses.  At  another  level,  one  teacher  set  out  to  develop  web-supplemented  materials,  such  as  web  quests,  for  her 
lower-level  English  students.  She  found  that  students'  interest  in  these  innovative  materials  seemed  to  decrease  the 
dropout  rate  in  her  course.  At  another  level,  it  was  hoped  that  teachers  could  develop  web-based  "courselets"  for  many 
content  areas  that  students  who  could  not  attend  school,  due  to  illness  or  suspension,  for  example,  could  complete  on 
their  own.  The  school  also  plans  to  develop  web-based  instruction  to  help  students  prepare  for  state-mandated  exams 
(Dwyer  & Boyle,  1999.) 

The  overall  goals  of  the  initial  plans  were  to  increase  access  to  instruction  through  these  many  means,  and  also 
to  decrease  instructional  costs.  During  the  strategic  planning,  the  district  evaluated  several  web-course  authoring 
programs  and  selected  WebCT  for  its  major  systems,  although  teachers  also  have  access  to  other  software. 

The  district  had  recently  passed  a bond  issue  to  fund  purchases  of  computer  hardware  and  software,  along  with 
significant  upgrades  of  the  district's  technology,  infrastructure,  however,  here,  as  elsewhere  in  our  public  schools,  there  is 
little  to  no  funding  for  teacher  training,  student  and  faculty  support,  and  development  of  materials. 

The  Collaborative  Partnership 

Meanwhile,  during  spring  of  2000,  at  a nearby  university,  a faculty  member  from  an  educational  technology 
graduate  program  sought  "real-world"  projects  for  students  in  her  advanced  instructional  development  class.  A 
fortunately-timed  call  to  the  technology  coordinator  at  the  high  school,  not  coincidentally,  an  alumnus  of  our  doctoral 
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program,  seeded  the  beginning  of  a collaborative  school-university  partnership  that  has  added  to  the  nucleus  of  the  web- 
based  school  project. 

Initially,  the  school  district  and  the  university  program  identified  several  needs  which  it  was  hoped  would  be 
answered  through  the  partnership.  The  school  district  hoped  to  add  a scholarly  focus  to  their  technology  efforts.  They 
also  hoped  the  graduate  students  would  provide  the  teachers  and  staff  with  an  instructional  design  perspective,  to 
complement  the  desired  exemplary  use  of  technology  skills.  It  was  hoped  that,  in  addition  to  the  extra  resources  the 
university  program  would  provide,  the  educational  technology  graduate  students  and  professor  would  bring  the  school 
and  the  district  a fresh  and  different  perspective. 

The  university  professor  and  students,  of  course,  hoped  to  fill  their  need  for  advanced  real-world  instructional 
design  projects,  as  well  as  a sort  of  proving  ground  for  the  graduate  students  in  which  they,  too,  would  learn  more  about 
school  needs  and  applications  of  technology.  Not  incidentally,  the  school  district  surrounds  the  university  and  this  close 
proximity  aided  all  in  staying  in  close  touch. 

Finally,  the  university  professor  and  the  school  technology  director,  hoped  that  the  personal  relationships  that 
developed  as  a result  of  the  partnership  would  sow  the  seeds  for  future  collaborative  projects  to  benefit  students  and 
faculty  in  both  the  school  district  and  at  the  university. 

Development  of  the  Project 

During  the  spring  of  2000,  five  students  in  an  advanced  instructional  development  course  learned  about 
principles  of  web-based  development  and  course  delivery  and  techniques  for  conducting  needs  assessments.  For 
instance,  they  learned  some  of  the  basic  principles  of  distance  learning,  especially  web-based  distance  learning  (cf. 
Mclsaac  and  Gunawardena,  1996;  Kahn,  1997;  Moore  & Kearsley,  1996.)  They  discussed  how  to  determine  if  a course 
is  suitable  for  web  delivery  by  learning  the  principles  of  Porter  (1997)  and  McManus  (1998.)  They  learned  about 
qualities  of  instructors  who  are  more  successful  in  developing  innovative  internet-based  programs  (cf.  Beaudoin,  1990; 
Collis,  1996;  Gunawardena,  1992;  Muffaletto,  1997.)  They  also  learned  about  aspects  of  learner  expectations  in  distance 
instruction  (cf.  Moore,  1989.) 

The  focus  of  the  course  then  shifted  to  developing  their  projects.  They  discussed  advanced  instructional  design 
techniques,  especially  instructional  strategies,  based  on  the  work  of  Smith  and  Ragan  (1999.)  They  also  learned  how  to 
implement  sound  instructional  design  principles  in  web-based  environments  (cf.  Eastmond  & Zieghan,  1995;  Hirumi  & 
Bermudez,  1996;  Ritchie,  & Hoffman,  1997.)  They  learned  about  some  of  the  issues  facing  teachers  and  students  who 
use  the  computer-mediated  communications  that  such  systems  as  WebCT  allow  (cf.  Bull,  Bull,  & Sigmon,  1997;  Lewis, 
Whitaker  & Julian,  1995.)  Students  were  reminded  of  the  importance  of  good  visual  design  in  web  development  (cf. 
Grabinger,  1989.)  Finally,  after  students  developed  their  projects,  they  field-tested  them  according  to  the  principles  of 
Dick  and  Carey  ( 1 996.) 

Four  projects  were  completed,  all  developed  in  partnership  with  teachers,  technology  coordinators  and  students. 
These  projects  will  be  presented  briefly  below. 

Rookie  Camp:  An  Introductory  Unit  for  Web-supplemented  Instruction  at  the  High  School  Level 

The  Rookie  Camp  unit  in  WebCT  provides  students  with  an  introduction  to  the  elements  they  will  be 
working  with  in  a web-supplemented  learning  environment  through  (Niemczyk,  2000.)  Ease  of  use  is  critical  to 
promoting  success  in  computer-assisted  instruction.  If  the  students  continually  need  help  in  overcoming  obstacles  in 
the  software,  they  will  become  frustrated  quickly  (Heinich,  Molenda,  Russell,  & Smaldino,  1998).  Though  it  is 
probably  true  that  many  high  school  students  have  some  computer  experience,  relatively  few  have  had  exposure  to 
using  the  web  as  part  of  their  classroom  instruction.  In  order  to  insure  success,  the  learners  need  to  be  ready  to  work 
with  this  new  tool.  Learner  readiness  involves  gaining  competencies  in  using  navigation  tools  and  becoming 
familiar  with  the  learning  environment  (Twigg,  1999;  Winiecki,  1999). 

In  order  to  ease  the  transition  to  web-supplemented  courses  and  promote  success,  the  Rookie  Camp  unit 
was  developed.  Rookie  Camp  focuses  on  the  tools  and  working  areas  of  WebCT  in  the  “default”  course  template. 
Explanation  of  the  course  homepage  depicting  nine  icons  is  described.  Each  of  the  nine  icons  and  the  course 
management  tools  they  represent  are  then  explained  in  detail.  After  the  students  work  through  the  Rookie  Camp 
course  content,  they  are  provided  practice  activities,  review  and  a quiz. 

This  unit  will  become  a “default  tool”  to  be  included  in  all  WebCT  courses  at  this  high  school.  Rookie 
Camp  will  be  the  first  unit  the  high  school  students  in  web-supplemented  courses  will  go  through.  (For  a more 
complete  description  of  this  project  see  Niemczyk,  Dwyer,  & Savenye,  2000.) 
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Poetic  Devices  and  the  Epic  Hero  in  the  Odyssey  of  Homer:  The  Odyssey  Project 

Two  graduate  students  (Kim  & Nicolaou,  2000)  developed  a web-based  unit  to  help  teachers  teach  The 
Odyssey . The  project  was  designed  to  be  used  both  as  a stand-alone  unit  and  a supplemental  unit  of  instruction, 
either  synchronously  or  asynchronously.  The  project  was  designed  to  meet  the  teacher’s  needs,  as  well  as  the  school 
partner’s  need  of  creating  a working  model  and  template  of  what  a web-based  course  should  look  like.  The  unit 
teaches  students  how  to  identify  several  poetic  devices  and  epic  hero  qualities  in  given  passages.  It  also  allows 
students  to  explore  an  interactive  map  of  Greece  and  the  Mediterranean  region,  with  references  to  events  in  The 
Odyssey . 

The  project  was  developed  over  a period  of  several  weeks,  and  was  used  and  tested  at  the  end  of  the  school 
partner's  spring  semester.  The  project  was  field  tested  with  eight  9th  grade  English  students.  Students  checked  off 
multiple-choice  test  items  on  a web  page,  and  then  submitted  their  answers  electronically  to  the  designers.  The 
courselet  included  an  online  student  attitude  survey  with  5-point  Likert-type  questions  and  open-ended  text  boxes. 
Students'  attitudes  were  very  favorable.  Students  reported  that  they  enjoyed  using  the  courselet  and  would  use  more 
courselets  like  The  Odyssey  unit.  Teachers’  attitudes  were  also  positive.  However,  student  achievement  was  rather 
poor  and  might  have  been  influenced  by  the  unfavorable  conditions  under  which  the  field  test  was  conducted 
(through  an  unfortunate  circumstance,  field  testing  had  occurred  at  the  same  time  as  state-wide  testing). 

Future  plans  include  a revision  to  include  more  descriptive  feedback  and  more  short  activities  sections  to 
give  students  more  opportunities  to  practice  what  they  have  learned.  More  graphics  will  be  added  to  the  instruction 
as  well.  The  most  important  lessons  learned  from  this  project  were  that  face-to-face  teacher-designer  meetings  need 
to  be  frequent  and  regular,  and  that  time  management  is  critical. 

Teacher  Training  and  Support 

One  doctoral  student  conducted  a study  to  determine  how  the  district  could  best  train  and  support  teachers  in 
using  web-based  instruction.  In  her  needs  assessment  study,  she  surveyed  the  literature,  researched  case  studies  on  the 
web,  and  conducted  structured  interviews  and  surveys  with  school  district  technology  coordinators,  teachers  and  the 
VHS  students  (Olina,  2000.) 

Olina  found  that  the  major  missing  component  in  most  technology-integration  projects  in  schools  has  been  the 
lack  of  adequate  teacher  training  and  support  (Office  of  Technology  Assessment,  1995;  President's  Committee  of 
Advisors  on  Science  and  Technology,  1997.)  Based  on  her  preliminary  work  with  the  school-district  personnel,  Olina 
determined  that  she  would  develop  a "white  paper"  that  would  further  inform  the  decision-making  process  regarding  the 
implementation  of  web-based  and  web-supplemented  instruction  at  the  high  school  and  across  the  district. 

Olina's  major  findings  include: 

The  most  common  barriers  to  technology  integration  in  teaching  are  the  increased  preparation  time,  a lack  of 
awareness  of  the  general  benefits  of  distance  education,  faculty  compensation  and  incentives,  access  to  appropriate 
technologies,  a lack  of  shared  vision  about  distance  education  in  the  organization,  institutional  barriers,  and  a lack 
of  support  staff  to  help  with  course  development  (Berge  & Muilenburg,  2000;  Moore  & Kearsley,  1996;  Office  of 
Technology  Assessment,  1995.) 

The  University  of  Illinois  Faculty  Seminar  Report  (1999)  highlights  the  need  for  teacher  training,  both  to  help 
faculty  adapt  their  pedagogical  styles  to  the  needs  of  online  teaching,  and  to  learn  their  roles  in  moderating  online 
class  interactions. 

It  is  advisable  to  focus  training  on  the  use  of  technology  on  teaching  itself,  rather  than  on  skills  using  software. 
Systemic  support  systems  should  be  developed  to  address  the  major  barriers  to  technology  integration  described 
above. 

A few  of  the  implications  described  in  Olina’s  white  paper  include: 

Develop  a clear  vision  for  technology  integration  that  is  directly  linked  to  improvement  of  classroom  practice. 

Set  up  a campus-wide  task  force  for  technology  integration. 

Allocate  adequate  financial  resources  for  teacher  professional  development. 

Use  a wide  range  of  professional  development  strategies  that  are  tailored  to  the  particular  needs  of  different  groups 
of  teachers.  (For  a more  complete  description  of  this  project  see  Olina,  Dwyer,  & Savenye,  2000.) 

Digital  Copyright  for  Teachers 

The  final  project  developed  as  part  of  the  partnership  concerned  helping  teachers  learn  about  copyright  issues 
related  to  web  development.  Kopp  (2000)  developed  a prototype  design  for  a web-based  self-instructional  unit  to  help 
teachers  learn  about  copyright  issues  in  the  digital  age.  Teachers  learn  how  to  identify  material  that  can  be  copyrighted, 
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how  to  secure  copyright  permission,  and  how  to  follow  guidelines  of  educational  use  of  copyrighted  material.  The 
prototype  materials  have  been  field-tested  with  teachers  and  are  currently  being  revised  for  dissemination. 

Implications 

With  the  explosive  growth  of  the  internet,  many  schools  and  universities  are  turning  to  the  internet  to  both 
supplement  traditional  course  materials,  in  web-supported  courses,  and  to  deliver  fully  web-based  courses. 

We  believe  collaborations  among  organizations  like  ours  hold  great  promise  for  building  and  developmental- 
testing  high-quality  learning  materials  for  students  and  support  for  teachers.  The  school  district  was  able  to  leverage  its 
resources  and  teachers'  time  to  work  with  university  personnel  and  complete  four  projects  for  students  and  teachers. 

In  developing  our  partnership  we  learned  many  lessons.  Among  them  are: 

People,  not  institutions,  build  relationships. 

The  needs  of  both  parties  must  at  all  times  be  attended  to. 

There  is  an  advantage  to  informal,  rather  than  formal,  partnerships. 

All  parties  need  to  have  a flexible  attitude. 

The  right  people  at  the  right  time  make  up  the  right  partnership. 

The  results  of  the  partnership  reflect  the  levels  of  involvement  of  all. 

Success  builds  over  time. 

In  the  future  we  plan  to  continue  the  partnership  with  a continuing  emphasis  on  informality  and  quality  rather 
than  quantity  of  projects  and  goals.  We  intend  to  expand  the  partnership  to  other  courses  and  programs  at  the  school 
district  and  university,  as  appropriate.  We  also  will  continue  to  explore  web-based  technologies  for  communication 
among  the  partners  and  to  look  for  increasingly  innovative  ways  to  partner. 
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Abstract 

In  recent  decades,  instructional  computer  programs  and  websites  have  become  increasingly  prevalent. 
Programmers,  academics,  and  free-lance  computer  professionals  have  all  begun  to  create  what  they  tout  to  be 
instructionally  sound  products  that  will  help  people  learn.  Many  of  these  products  however,  are  rarely  evaluated  on 
any  level  for  various  facets  of  their  design.  When  they  are  evaluated,  it  is  usually  either  to  gather  information  about 
the  navigability  or  functioning  of  the  hardware/software,  or  even  less  frequently,  to  assess  how  well  people  have 
acquired  the  skills  and  knowledge  taught  in  the  program. 

As  the  value  of  media-delivered  information  is  increasingly  emphasized  as  a powerful  instructional  tool, 
user  knowledge  gains  are  becoming  more  frequently  investigated.  Formative  evaluation  techniques  that  assess 
learning  are  often  employed.  While  it  is  important  to  appraise  this  aspect  of  an  instructional  program  or  website,  it 
is  also  necessary  to  investigate  the  functionality  of  the  electronic  characteristics  of  the  same  program.  Heuristic 
evaluation  techniques  are  rapidly  becoming  the  evaluation  method  of  choice  to  assess  this  aspect  of  instructional 
programs.  Unfortunately,  the  two  areas  - functionality  and  learning  - are  rarely  both  assessed  within  the  same 
evaluation  due  to  time,  money,  and  methodological  constraints. 

While  the  growing  popularity  of  media-based  instructional  programs  has  advanced  instructional  design 
and  development  techniques,  equally  efficient  and  effective  evaluation  methodologies  to  correspond  to  this  new 
manner  of  instructional  design  have  fallen  behind.  Current  evaluation  techniques  for  electronically  delivered 
instruction  are  either  poor  in  their  methodology  or  incomplete  in  their  design.  Often,  all  pertinent  aspects  of 
instructional  programs  are  not  assessed  and  the  learner  is  left  with  only  a partially- sound  form  of  instruction.  A 
new  type  of  evaluation  therefore  needs  to  evolve  to  keep  current  with  new  design  and  development  strategies.  A 
hybrid  of  heuristic  and  formative  evaluation  is  proposed. 

" Evaluation  is  a discipline  inquiry  to  gather  facts  and  other  evidence  that  allow  an  evaluator  to  make 
assertions  about  the  quality,  effectiveness  or  value  of  a program,  a set  of  materials,  or  some  other  object  of  the 
evaluation  in  order  to  support  decision  making ” (Cummings,  1998).  As  such,  evaluation  techniques  and 
methodologies  exist  in  many  different  fields,  and  are  performed  on  a wide  variety  of  materials,  programs,  and 
products.  While  evaluation  in  its  purest  concept  adheres  to  Cummings  (1998)  definition,  particular  types  of 
evaluation  are  performed  with  specific  goals  in  mind,  using  unique  methodologies.  Heuristic  and  formative 
evaluations  are  among  the  two  more  prevalent  methods  used  in  the  field  of  educational  technology  at  present. 

Heuristic  Evaluation 

Heuristic  evaluation  is  a type  of  usability  testing.  Usability  testing  has  its  roots  in  classical  experimental 
methodology  (Rubin,  1994)  and  has  experienced  popular  application  in  the  field  of  engineering.  It  is  a systematic 
way  of  evaluating  the  functionality  of  a product  (usually  electronic)  by  observing  users  and  recording  information 
about  areas  of  difficulty  and  ease  within  a program  (Dumas  & Redish,  1993).  Dumas  and  Redish  (1993)  describe 
five  characteristics  of  every  usability  test  as  follows:  1)  the  primary  goal  is  to  improve  product  usability,  2) 
individual  usability  tests  also  have  unique  goals  that  are  determined  based  on  specific  needs,  3)  testing  evaluators 
are  actual  users,  4)  testing  evaluators  perform  authentic  tasks,  5)  problem  areas  are  revealed  through  data  analysis 
and  modifications  are  recommended. 

Commonly,  usability  testing  is  not  implemented  in  its  purest  form  (Nielsen  1993;  Whiteside,  Bennett,  & 
Holtzblatt,  1988).  Costs  for  full-scale  usability  tests  are  perceived  to  be  very  prohibitive  (Nielsen,  1994),  and  their 
methodologies  very  complex  (Belotti,  1988).  A simplified  method  of  usability  was  therefore  developed  by  Nielsen 
in  1989  called  "discount  usability  engineering"  (Nielsen  1989b,  1990a,  1993),  One  of  the  most  popular  types  of 
discount  usability  engineering  is  termed  "heuristic  evaluation." 
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Heuristic  evaluation  (Nielsen  1994)  engages  a small  set  of  evaluators,  usually  three  to  five,  to  examine  an 
interface  and  assess  its  adherence  to  a prespecified  set  of  usability  criteria,  or  "heuristics".  Each  evaluator 
progresses  through  a program  individually  and  either  records  their  findings  in  writing  or  verbalizes  them  to  an 
observer  who  is  present  during  each  session.  If  an  observer  is  present,  he/she  is  also  allowed  to  assist  the  evaluator 
in  navigating  the  site  if  necessary  or  to  answer  other  questions  that  may  arise.  A debriefing  session  may  also  occur 
at  the  end  of  each  evaluation  session  to  gather  more  information.  After  all  the  evaluators  have  completed  their 
sessions,  the  data  is  then  analyzed  and  items  needing  to  be  revised  are  identified. 

Formative  Evaluation 

In  many  respects,  heuristic  evaluation  is  not  unlike  formative  evaluation  as  classically  defined  in  the  . 
educational  technology  literature.  Coined  in  1967  by  Michael  Scriven,  the  term  formative  evaluation  is  viewed  as  a 
means  of  identifying  areas  of  modification  in  the  development  of  educational  materials  through  the  collection  and 
analyzation  of  data  from  the  target  population.  This  is  different  from  "summative  evaluation"  which  occurs  after 
development  in  order  to  determine  effectiveness  (Smith  & Ragan,  2000). 

Dick  and  Carey  (1996)  propose  three  phases  to  formatively  evaluating  instructional  materials.  These  are 
the  one-to-one  or  clinical  evaluation,  the  small-group  evaluation,  and  the  field  trial.  The  one-to-one  evaluation  stage 
occurs  individually  with  one  to  three  learners  who  are  representative  of  the  target  population.  Ideally  the  three 
consist  of  one  high-ability,  one  medium-ability  and  one  low-ability  learner.  The  one-to-one  evaluation  is  utilized  in 
order  to  identify  any  factual  errors  in  the  instruction  and  to  obtain  initial  reactions  and  indications  of  performance 
improvement.  Questionnaires  regarding  learner  attitudes  are  generally  used  as  the  main  data  collection  instrument. 

Once  the  instructional  materials  have  been  revised  according  to  the  information  gathered  from  the  one-to- 
one  evaluation,  a small  group  evaluation  should  be  performed  with  approximately  eight  to  twenty  learners.  Again, 
these  learners  should  be  representative  of  the  target  population  as  much  as  possible.  Learners  should  be  selected  at 
random  so  that  your  results  can  be  generalized  to  the  entire  population.  Two  primary  purposes  for  the  small  group 
evaluation  are  to  1)  determine  the  effectiveness  of  changes  made  following  the  one-to-one  evaluation,  and  2) 
identify  any  remaining  learning  problems  that  learners  may  have.  In  this  phase,  learner  performance  scores  on 
pretests  and  posttests  are  typically  used  to  evaluate  instructional  effectiveness.  Attitudes  toward  the  instruction  are 
evaluated  through  questionnaires  or  follow-up  interviews. 

The  field  trial  is  the  final  formative  evaluation  phase  that  Dick  and  Carey  (1996)  discuss.  It  involves  the 
participation  of  a randomly  sampled  group  of  about  thirty  individuals  who  are  representative  of  the  target 
population.  It  is  used  to  determine  the  effectiveness  of  the  changes  resulting  from  the  small  group  evaluation  and 
whether  the  instruction  can  be  used  in  the  context  for  which  it  is  intended.  Much  like  a dress  rehearsal,  it  provides 
the  last  chance  to  identify  and  remove  any  remaining  errors  or  problems.  There  are  many  similarities  between  the 
field  trial  and  the  small  group  evaluation.  The  main  difference  between  the  two  evaluation  processes  is  in  the  actual 
authenticity  of  the  materials,  learners,  procedures,  instructors  and  setting.  The  field  trial  should  mirror  the  intended 
instructional  experience  as  much  as  is  possible. 

Several  similarities  exist  between  heuristic  and  formative  evaluation.  Some  experts  consider  formative 
evaluation  to  be  the  underlying  blueprint  for  heuristic  evaluation  (Hix  & Hartson,  1994).  The  two  methodologies 
also  appear  to  have  similar  goals.  They  both  use  data  collected  from  a target  population  in  order  to  make 
recommendations  regarding  modifications  to  a specific  product  or  material  in  the  design  and  development  phases  of 
creation.  Both  employ  the  use  of  surveys,  observations,  interviews,  and  various  other  data  collection  instruments 
and  techniques.  However,  despite  apparent  similarities,  there  do  exist  fundamental  differences  between  the  two. 
Formative  evaluation  primarily  focuses  instructional  and  learning  strategies,  as  evidenced  by  the  use  of  pretests  and 
posttests.  Heuristic  evaluation  concentrates  more  on  the  usefulness  of  a product,  i.e.  the  user  interface,  navigation 
issues,  etc. 

In  1987,  Patterson  and  Bloch  called  for  formative  evaluation  to  be  conducted  during  the  development  of 
computer-assisted  instruction  (CAI).  In  their  article,  they  propose  investigating  learning  gains,  user  attitudes, 
interface  and  navigation  issues  utilizing  Dick  and  Carey’s  (1996)  three  phases  of  formative  evaluation  as  a structure. 
These  areas  of  investigation  within  educational  media  products  still  remain  of  utmost  importance  today.  However, 
with  the  advent  of  the  Internet  and  the  World  Wide  Web  (WWW),  as  well  as  the  ever-increasing  rapidity  with 
which  electronic  educational  products  are  produced,  the  implementation  of  the  methodology  Patterson  and  Bloch 
(1987)  propose  is  impractical  in  the  currently  fast-paced  realm  computerized  instruction. 

A methodology  that  would  perhaps  serve  the  needs  of  efficiency,  practicability,  and  still  investigate 
learning,  attitudes,  interface  and  navigation  therefore  needs  to  be  created.  The  present  work  attempts  to  address 
such  a problem  with  the  development  and  testing  of  a new  method.  It  consists  of  a combination  of  heuristic  and 
formative  evaluation  techniques.  In  1 997,  Corry,  Frick,  and  Hansen  incorporated  usability  testing  techniques  into 
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the  development  of  university  informational  website.  They  used  many  facets  of  heuristic  evaluation  as  the  structure 
of  their  investigation.  According  to  their  results,  this  was  a very  efficient  method  for  exploring  interface,  navigation, 
content  and  some  attitudinal  issues.  Learning,  however,  could  not  be  measured  as  the  site  was  informational  and  not 
instructional. 

The  purpose  of  the  present  study  was  to  further  investigate  the  use  of  heuristic  evaluation,  in  conjunction 
with  formative  evaluation,  as  a methodology  for  assessing  instructional  websites.  Through  this  type  of  mixed 
methodology,  not  only  can  attitudes,  user  interface,  and  navigation  issues  be  explored,  learning  and  instructional 
strategies  can  be  investigated  as  well.  An  evaluation  of  this  nature,  illustrating  the  proposed  methodology,  is 
described  in  the  case  study  that  follows. 

Method 

Participants 

Participants  in  the  study  were  five  students  at  an  urban  university  in  or  prior  to  their  first  semester  in  a 
graduate  programme  in  education.  All  participants  will  soon  be  formally  enrolled  in  a required  Human  Performance 
Technology  course  and  participated  in  the  study  because  the  instructional  program  used  in  it  covers  content 
regularly  taught  in  the  course.  Although  the  site  content  was  required  course  material,  the  evaluation  sessions  were 
several  hours  long  and  it  was  decided  that  evaluators  would  be  paid  a small  stipend  for  their  time. 

Materials 

The  instructional  materials  were  designed  to  teach  various  Human  Performance  concepts,  including  needs 
analysis,  instructional  design,  and  formative  and  summative  evaluation.  Actual  course  content  and  activities  were 
adapted  into  an  interactive  instructional  website  for  the  study.  The  site  was  created  initially  with  the  intention  of 
supplementing  an  on-site  course.  The  long  term  goal  of  the  website  is  to  have  it  serve  as  a stand-alone,  distance- 
delivered  web  course. 

The  site  contained  a total  of  10  instructional  modules,  a homepage  (including  the  course  syllabus  and 
navigation  information),  and  a sitemap.  Eight  of  the  ten  modules  contained  multiple-choice  and/or  constructed- 
response  practice-with-feedback  activities.  The  remaining  two  modules  were  solely  informational.  For  the  current 
purposes,  the  site  is  supplemental  to  the  on-site  Human  Performance  Technology  course.  In  future,  it  will  be  used  as 
the  basis  for  a distance  version  of  the  same  course.  The  current  URL  for  the  site  is 
http://doe.concordia.ca/etec5 1 2_7 1 2/index.html 

The  website  contained  three  overall  course  objectives  and  23  learning  objectives.  16  of  these  objectives 
were  short-term  and  to  be  completed  within  the  site,  while  seven  of  the  objectives  were  subobjectives  of  the  long- 
term course  objectives.  Each  of  the  objectives  was  taught  through  a number  of  screens  which  presented  instruction, 
five  practice-with-feedback  items  (for  the  short-term  objectives),  summaries,  and  reviews.  Seven  objectives 
required  selected  responses  in  a multiple-choice  format  and  nine  required  constructed  responses.  Practice  items 
consisted  of  multiple-choice  questions  with  two-to-four  response  choices  for  the  seven  selected-response  objectives. 
For  the  constructed-response  items,  participants  typed  their  answer  in  a field  and  pressed  a submit  button.  In  the 
next  field  a sample  answer  appeared  to  which  participants  compared  their  answer.  The  site  was  unable  to  track  each 
participant's  progress  by  recording  response  choices  due  to  the  inability  of  the  university  server  to  support  the 
necessary  interface. 

Learners  could  advance  through  the  site  by  selecting  links  to  modules  from  the  menu  or  site  map.  Once  in 
a module,  they  could  choose  to  view  any  screen  by  selecting  a link  from  a table  of  contents,  or  they  could  simply 
advance  in  a linear  fashion  by  clicking  "previous"  or  "next"  buttons  that  appeared  on  each  screen.  As  the  site  was 
programmed  to  work  on  either  a PC  or  a Macintosh  platform,  learners  were  also  able  to  progress  through  the  site 
using  the  navigation  options  within  the  web-browser  they  were  utilizing. 

Procedures 

Prior  to  engaging  in  the  evaluation  of  the  instructional  website,  it  was  ascertained  that  participants  were 
graduate  students  in  the  department  of  education,  who  were  enrolled  in  a degree  programme.  The  participants  also 
must  not  have  taken  the  Human  Performance  Technology  course  prior  to  evaluating  the  website.  These  were  the 
only  prerequisites  necessary  for  learners  to  be  able  to  evaluate  the  website.  Permission  was  given  by  the  participants 
to  video-tape  each  evaluation  session  in  case  the  tapes  of  the  sessions  had  to  be  reviewed  for  data  gathering 
purposes. 

During  each  session,  each  participant  was  then  stationed  at  a computer  terminal  and  given  access  to  the 
website.  He  or  she  was  presented  with  a list  of  heuristics  and  instructed  to  record  errors  and  respond  to  the  various 
heuristics  while  progressing  through  the  site.  Participants  were  also  asked  to  verbalize  their  thoughts  and  questions 
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while  utilizing  the  program  (Smith  & Wedman,  1988).  A researcher  sat  beside  each  participant,  recording 
observations  as  he/she  advanced  through  the  site.  The  researcher  answered  any  questions  that  arose 
regarding  navigation  or  content,  although  he  redirected  the  participants  to  try  to  answer  content-related  questions  by 
referring  to  the  site.  Also,  the  participants  were  asked  questions  in  response  to  their  comments  in  order  to  probe 
more  deeply  into  the  nature  of  participant  statements.  Full  learner  control  was  given  to  the  participants  due  to  the 
fact  that  they  could  progress  through  the  site  at  any  pace  they  chose,  viewing  any  screens  at  any  point,  with  the  only 
restriction  being  that  they  must  complete  all  the  practice  activities  that  correspond  to  the  short-term  objectives. 

As  the  stated  previously,  the  university  server  would  not  support  any  interface  that  would  allow  tracking  or 
recording  of  participant  progression  through  the  site,  so  another  method  was  devised  to  record  student  responses  to 
the  practice  items.  After  each  practice  set,  the  researcher  copied  and  pasted  participant  answers  to  the  practice  items 
into  a separate  word  document.  Correct  or  incorrect  answers  to  the  multiple-choice  items  were  indicated, 
while  statements  from  the  participants  as  to  the  correctness  of  their  answers  in  comparison  to  the  sample  answers  for 
the  constructed-response  items  were  also  recorded. 

Overall,  it  took  each  participant  approximately  1 1 hours  to  complete  the  evaluation  of  the  website.  As  the 
evaluation  was  time  consuming,  participants  completed  the  activity  in  several  sessions  which  varied  in  length 
according  to  their  individual  preferences  and  schedules.  Upon  completion  of  the  evaluation,  each  participant  was 
interviewed  in  a short  debriefing  session  to  get  any  last  impressions  or  thoughts  he  or  she  may  want  to  communicate. 
A paper-and-pencil  survey  regarding  attitudes  toward  the  website,  perceived  future  usefulness,  and  areas  of 
modification  was  also  administered  at  this  time  and  participants  returned  the  survey  prior  to  leaving  the  session. 

Criterion  Measures 

Learner  achievement  was  measured  through  performance  on  the  practice  items.  The  35  multiple-choice 
items  were  scored  either  one  or  zero,  and  the  45  constructed-response  items  were  scored  either  two  (completely 
correct  answer),  one  (partially  correct  answer),  or  zero  (no  answer  or  incorrect  answer)  according  to  a scoring  key 
developed  by  the  experimenters.  Thus,  the  maximum  possible  score  on  the  practice  items  was  125. 

The  32-item  attitude  questionnaire  assessed  participants’  satisfaction  with  the  material,  their  perceived 
future  usefulness  of  the  skills  learned  in  the  site,  and  suggestions  for  modifications.  The  attitude  questionnaire, 
containing  12  three-choice  Likert-type  items,  seven  yes/no  items,  and  13  constructed-response  items,  was 
administered  immediately  after  participants  completed  the  instructional  program.  A sample  item  from  the  attitude 
questionnaire  is  below: 

Circle  how  much  you  liked  each  of  the  activities  listed  below. 

4.  Receiving  feedback  that  asked  you  to  I liked  I liked  I did  not 

compare  your  answers  to  a sample  this  a lot  this  OK  like  this 

answer. 

Responses  to  the  list  of  heuristics  were  open-ended.  There  were  five  catagories  in  which  to  respond, 
including  site  content,  site  navigation,  graphical  appropriateness,  readability,  and  communication  venues.  A sheet 
where  general  errors  were  recorded  was  included.  Sample  heuristics  are  presented  below: 

Please  provide  specific  feedback  (positive  and/or  negative)  on  the  HPT  website  regarding  the  following: 

• Site  content  (information,  samples/examples,  practice  items,  answers  to  practice  items,  links/resources) 

• Site  navigation  (interface,  platform  conventions,  i.e.  buttons,  etc.,  menubar,  site  map,  navigation  instructions) 

Researcher  observations  to  the  think-aloud  protocols  were  recorded  per  individual  participants,  per  session. 
Observations  were  made  in  a variety  of  catagories.  These  catagories  were  quite  similar  to  the  heurstics  and  included 
site  content,  site  navigation,  graphical  appropriateness,  readability,  and  communication  venues.  Observations  on 
learning  and  attitudes  were  also  recorded. 

Data  Analysis 

Calculation  of  simple  mean  scores  for  the  practice  items,  individually  and  collectively,  were  tabulated  to 
indicate  achievement.  Responses  to  the  attitude  survey  were  tallied  and,  for  the  open-ended  items,  categories  of 
responses  were  created.  Categories  of  reponses  were  also  created  to  calculate  data  on  the  debriefing  responses, 
heuristic  lists  and  researcher  observations. 

Results 

Achievement 

Mean  overall  practice  item  scores  per  instructional  objective  are  shown  in  Table  1.  Within  the 
“Interventions”  module  of  the  Web  site,  participants  scored  differently  on  the  two  module  objectives.  Although 
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participants  scored  relatively  high  when  asked  to  classify  performance  solutions  as  instructional  or  non-instructional 
using  a forced-choice  format  (84%),  they  scored  significantly  lower  (46%)  when  prompted  to  offer  possible 
solutions  to  given  case  scenarios  using  an  open-ended  question  format.  Another  noteworthy  result  occurred  within 
the  “Practice  and  Feedback”  module  of  the  Web  site.  When  prompted  to  identify  appropriate  practice  activities  for 
given  objectives  using  a forced-choice  format,  a perfect  mean  overall  score  occurred  (100%).  Asked  to  write 
appropriate  practice  activities  for  given  objectives,  overall  participant  score  lowered  significantly  (60%).  The  trend 
of  participants  scoring  highly  on  closed-ended  questions  does  not  continue  with  the  practice  items  score  from  the 
“Sequencing”  module.  Participants  were  presented  with  various  course  or  workshop  and  asked  to  identify  their 
pedagogical  components.  Participants  answered  all  prompts  incorrectly,  resulting  in  an  overall  mean  score  of  zero 
(0%).  The  overall  percentage  scores  of  participants  ranged  from  61.6%  to  68.8%,  with  a mean  overall  percentage 
score  of  65.3%. 

Table  1 

Mean  Overall  Practice  Item  Scores  per  Instructional  Objective 


Mean 


Instructional  Objectives  Score 

£%) 

Classify  performance  solutions  as  instructional  or  non-instructional.  84% 

Offer  possible  solutions  to  given  case  scenarios.  46% 

Develop  sections  of  responses  to  given  proposal  scenarios.  58% 

Given  various  data  sets  from  a needs  analysis,  develop  recommendations  based  on  your  conclusions  68% 

from  the  data. 

Develop  sample  data  collection  items  from  given  case  scenarios.  68% 

Identify  well-written  instructional  objectives  . 92% 

Write  instructional  objectives.  66% 

Identify  appropriate  assessment  items  for  given  instructional  objectives.  88% 

Identify  well-written  assessment  items.  72% 

Write  appropriate  assessment  items  for  instructional  objectives.  80% 

Identify  appropriate  practice  activities  for  given  objectives.  100% 

Write  appropriate  practice  activities  for  given  objectives.  60% 

Identify  the  pedagogical  components,  given  various  courses  or  workshops.  0% 

Determine  whether  the  evaluations  described  in  given  scenarios  are  formative  or  summative . 92% 

Design  a methodology  and  state  the  instruments  to  be  used  for  the  type  of  evaluation  indicated  in  given  76% 

scenarios. 

Note:  Total  number  of  Participants  = 5. 


Attitudes 

When  asked  to  indicate  liking  of  activities,  participants  responded  positively  to  receiving  feedback  that 
asked  to  compare  their  answers  to  sample  answers,  with  a majority  of  participants  (80%)  saying  they  liked  the 
activity  “a  lot”.  When  asked  to  indicate  the  importance  of  activities,  all  participants  (100%)  answered  that  both  the 
reading  material  presented  and  relating  the  information  in  the  Web  site  to  future,  practical  applications  was 
extremely  important.  A majority  of  participants  (80%)  answered  that  completing  the  practice  exercises  in  the 
module  were  extremely  important. 

All  participants  (100%)  did  not  believe  the  Web  site  was  too  hard  for  them  to  understand  and  complete.  All 
participants  (100%)  also  responded  that  they  were  able  to  successfully  compare  their  answers  to  the  sample  answers 
and  they  learned  important  techniques  that  would  be  of  value  in  the  real  world.  A majority  of  participants  (80%) 
found  the  practice  exercises  helpful,  were  able  to  successfully  navigate  the  Web,  relate  the  information  presented  to 
previous  learning  experiences  and  felt  they  would  be  able  to  apply  what  they  learned  in  the  Web  site  to  a real  world 
setting.  When  prompted  for  the  most  preferred  topic  of  the  Web  site,  participants  mentioned  the  “Needs 
Assessment”  module  the  most  stating  it  was  relevant,  informative  and  easy  to  understand.  When  prompted  for  the 
least  preferred  topic,  participants  mentioned  the  “Objectives  and  Assessment”  module  the  most,  stating  the  practice 
items  as  the  deciding  factor. 
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Heuristic  Commentaries 

Participant  commentaries  per  heuristic  are  shown  in  Table  2.  When  asked  to  remark  about  the  content  of 
the  Web  site,  participants  mentioned  its  unclear  and/or  confusing  wording  the  most  (31%  of  total  heuristic 
comments  made).  Despite  this,  participants  also  stated  that  the  Web  site  was  clear  and  useful  information  was 
provided  (25%  of  total  heuristic  comments  made).  Participants  were  satisfied  with  navigation  issues  (32%  of  total 
heuristic  comments  made).  The  graphics  and  reading  level  was  judged  as  appropriate  for  the  target  audience  (52%  of 
total  heuristic  comments  made).  When  prompted  for  errors  within  the  Web  site,  the  majority  of  statements  (54%  of 
total  heuristic  comments  made)  indicated  grammatical  errors  in  the  content.  The  grammatical  errors  may  also  be  a 
factor  in  participants  stating  that  the  content  was  confusing  (19%  of  total  heuristic  comments  made). 

Table  2 

Participant  Commentaries  per  Heuristic 


Heuristic 

Number  of  Responses 

Content 

Unclear  or  confusing  wording. 

(10) 

Clear  and  useful  information  provided. 

(8) 

Examples  with  feedback  very  helpful. 

(6) 

Inconsistent  presentation  of  content. 

(4) 

Insufficient  information  provided. 

(4) 

Navigation 

Satisfied  with  navigation  issues. 

(8) 

Inconsistent  presentation  of  content 

(5) 

Poor  design  issues. 

(4) 

Attractive  site  layout. 

(2) 

Non-working  functions. 

(1) 

Graphics  and  Reading  Level 

Appropriate  reading  level. 

(ID 

Text  cut  off  on  screen. 

(4) 

Insufficient  graphics. 

(2) 

Confusing  language  used. 

(2) 

Poor  design  detracts  from  learning. 

(2) 

Communication  Avenues 

Hyperlinks  useful. 

(5) 

Sufficient  for  site. 

(4) 

Miscellaneous 

(3) 

Other 

Miscellaneous 

(6) 

Sufficient  practice  and  feedback. 

(3) 

Informative  content. 

(1) 

Poor  design. 

(1) 

Poorly  written  content. 

(1) 

Errors 

Grammatical  errors. 

(60) 

Confusing  content. 

(21) 

Formatting  errors. 

(11) 

Layout/design  errors. 

(8) 

Miscellaneous 

(5) 

Non-working  functions. 

(2) 

Observations 

When  prompted  for  comments  related  to  navigation  of  the  Web  site,  many  related  to  the  poor 
interface/layout  design  of  the  Web  site.  Participants  were  also  vocal  about  the  poor  presentation  of  the  content  when 
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asked  about  the  graphics  and  reading  level.  Many  comments  about  the  communication  avenues  within  the  Web  site 
mentioned  the  its  high-level  access  to  information  (within  the  site  and  links  to  external  information  sources.)  The 
majority  of  comments  relating  to  errors  in  the  site  related  to  inconsistent  design  features  and  confusing  presentation 
of  its  content.  When  asked  about  the  learning  and  instruction  aspects  of  the  Web  site,  most  comments  referred  to  the 
relevant  practice  and  feedback  examples. 

Recommendations 

Data  from  achievement  on  the  practice  items  indicate  that  participants  learned  approximately  65%  of  the 
content  included  in  the  website.  This  would  indicate  that  while  the  website  appears  to  be  an  effective  supplement  to 
a course  in  human  performance  technology,  it  cannot  at  the  present  time  be  used  as  a stand-alone  distance  delivered 
web  course.  An  investigation  of  scores  per  objective  reveals  that  while  participants  generally  attained  successful 
scores  on  the  selected-response  practice  items,  scores  were  marginally  lower  on  the  constructed-response  questions. 
Participants  scored  below  60%  on  objectives  where  they  had  to  offer  solutions  to  case  scenarios,  develop  proposal 
sections,  and  write  practice  activities.  The  lowest  achievement  score  however  occurred  where  participants  had  to 
identify  pedagogical  components  of  an  instructional  design  package.  Attitude  responses  indicated  that  the  directions 
for  this  section  were  unclear  and  that  may  account  for  the  0%  success  rate  with  these  items.  Conversely,  attitude 
data  also  revealed  that  participants  thought  they  were  quite  successful  in  comparing  their  answers  to  given  sample 
answers.  This  may  mean  that  they  thought  they  scored  well  on  these  items  but  actually  did  not,  or  that  they  were 
successful  in  estimating  that  the  answers  they  created  were  incorrect  compared  to  the  sample.  What  is  clear  from 
this  data  is  that  several  of  the  constructed-response  practice  items  need  to  be  modified  and  that  the  directions  for  the 
practice  items  on  pedagogical  components  need  to  be  clarified. 

Participant  attitude  data  indicated  that  overall,  participants  like  the  site  and  thought  that  many  of  the 
activities  in  which  they  engaged  were  important.  As  well,  participants  responded  that  the  material  they  learned 
would  be  useful  in  a “real  world  setting”  and  the  skills  they  learned  could  be  transferred  to  situations  outside  of  the 
web  environment.  This  data  is  revealing  regarding  future  student  potential  to  engage  in  learning  the  information  in 
the  website.  Distance-delivered  web-courses  classically  suffer  high  attrition  rates.  One  factor  that  this  is  attributed 
to  is  motivation  level.  If  students  appear  to  like  the  website  described  in  this  study,  as  well  as  see  value  and 
transferability  in  the  material,  then  it  is  likely  that  at  least  motivationally,  the  site  would  be  a success  if  it  is  used  as  a 
stand-alone  course.  As  a course  supplement,  attitude  data  indicates  that  it  is  more  than  engaging  motivationally. 

Heuristic  response  data  was  successful  in  indicating  various  errors  within  the  website.  The  specific  errors 
were  not  indicated  here  as  they  were  not  deemed  of  interest  or  import  to  anyone  beyond  the  developers.  What  is  of 
interest  however  is  the  fact  that  the  technique  of  using  five  evaluators  was  successful  in  finding  and  documenting 
various  website  errors.  Further,  more  general  heuristic  data  revealed  that  while  the  reading  level  within  the  site  was 
appropriate,  the  way  certain  concepts  were  presented  was  confusing  and  needs  to  be  clarified  for  future  use. 
Navigation  issues  did  not  appear  to  be  of  great  concern  as  participants  responded  positively  to  the  methods  and 
venues  provided  for  navigation  within  the  site. 

Finally,  participant  think-aloud  responses  further  iterated  the  need  for  clarity  of  language  within  the  site  by 
requesting  more  examples  and  information  on  presented  concepts.  Participants  also  requested  that  more  graphics 
and  charts  be  included  in  the  site.  It  could  be  presumed  that  this  is  for  visual  appeal,  but  it  would  also  be  that  the 
increased  level  of  appropriate  charts  and  graphics  will  facilitate  further  explanation  and  clarity  of  concepts.  Finally, 
while  heuristic  data  revealed  no  problem  areas  within  navigation,  think-aloud  response  data  indicated  a modification 
to  the  menubar  and  “previous”  and  “next”  buttons.  Participants  st  ated  that  page  numbers  as  well  as  modules  should 
be  included  on  the  main  menubar  to  more  easily  enable  users  to  move  from  one  point  in  the  site  to  the  next. 

Overall,  it  is  clear  that  with  some  modification,  the  current  website  for  this  study  will  be  an  appropriate 
supplementary  source  for  an  onsite  course  in  human  performance  technology.  However,  issues  with  respect  to  user 
learning  gains  need  to  be  addressed  before  the  site  can  be  utilized  as  a distance  delivered  course.  Further,  the 
evaluation  methodology  utilized  within  this  project  appears  to  be  highly  successful  in  indicating  navigation, 
attitudinal,  informational  and  motivational  issues  within  instructional  websites.  It  is  however  still  not  clear  as  to 
how  accurately  learning  gains  can  be  measured  using  the  same  methodology.  Further  investigation  of  the 
methodology  altering  the  number  of  participants  used  to  measure  learning  gains  is  recommended  in  the  future. 
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Abstract 

This  paper  reports  the  results  of  a five-year  case  study  of  the  use  of  online  tools:  Internet,  e-mail , and  the 
WWW,  within  a Graduate  School  of  Education.  The  conceptual  framework  was  independently  developed,  but 
because  of  the  striking  parallel  with  activity  theory,  activity  theory  became  the  overall  framework  for  interpreting 
findings.  Ten  research  questions  were  investigated  using  multiple  surveys;  interviews  of  faculty,  staff,  and  students; 
a focus  group;  and  an  analysis  of  electronic  artifacts.  Principal  findings  included  the  following : 

• Self-efficacy  x perceived  value  persisted  across  time  and  across  programs  as  success  facilitators. 

• Personal/cultural  compatibility,  rather  than  time,  separated  earlier  from  later  adopters. 

• "Finding  a voice  and  having  something  to  say”,  a factor  identified  under  various  names  by  other  researchers, 
posed  a barrier  for  students  and  faculty  alike. 

• Users  valued  personal  scaffolding  but  had  individual  preferences  concerning  specific  types  of  scaffolding. 


Theory  Base 

This  case  study  investigated  the  factors  that  affect  the  use  of  the  Internet  within  a Graduate  School  of 
Education  (SOE).  In  this  context,  "Internet  tools"  were  defined  as  e-mail,  a FirstClass  ™ BBS  known  as  Colorado 
Education  On-line  (CEO),  and  the  WWW.  The  study  combined  both  empirical  research  within  the  school  and  an 
extensive  review  of  relevant  literature  to  identify  28  distinct  factors.  These  factors  grouped  into  six  naturally 
emerging  clusters  that  influence  the  diffusion  of  the  Internet  within  the  school.  (See  Figure  1.)  These  clusters  acted 
as  a starting  point  for  formulating  a new  model  of  adoption  of  telecommunications  by  institutions  of  higher 
education. 

Figure  I.  Clusters  of  Factors  Influencing  Diffusion  of  the  Internet 
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It  is  interesting  to  note  the  striking  parallel  between  Engestrom's  (1996)  depiction  of  an  activity  system  and 
the  process  that  may  be  occurring  in  the  SOE  regarding  the  adoption  and  use  of  the  Internet  as  the  activity  under 
investigation. 

A key  point  in  Soviet  psychology,  attributed  to  Vygotsky,  is  the  emphasis  on  the  use  of  tools  in  the 
development  of  human  mental  processes.  "The  tool  is  not  simply  added  on  to  human  activity;  rather,  it  transforms  it" 
(Tikhomirov,  1981:  270).  Engestrom  expands  Vygotsky’s  notion  to  conceptualize  human  activity  as  an 
interdependent  system  that  ties  the  individual  to  the  larger  cultural  context:  "Collective  activity  is  realized  through 
individual  actions,  but  it  is  not  reducible  to  a sum  total  of  those  actions"  (Engestrom,  1996:  262). 

In  Engestrom's  conceptual  framework,  known  as  an  activity  system,  the  activities  in  which  an  individual 
engages  tend  to  connect  six  elements,  namely:  (a)  the  individual  actor,  (b)  the  object  of  action  together  with  an 
expected  outcome,  (c)  the  tools  used  to  carry  out  the  activity,  (d)  the  community  of  which  the  actor  is  a part,  (e)  the 
norms  and  conventions  of  use  of  those  tools,  and  (f)  the  division  of  labor  that  characterizes  individual  actions  within 
local  collective  activities.  These  elements  are  all  interrelated;  changing  one  will  invariably  affect  the  rest  of  system. 
The  clusters  of  factors  in  Figure  1 can  be  loosely  identified  with  the  six  elements  of  an  activity  system.  This 
comparison  is  presented  in  Table  1 . 

Table  L A Comparison  of  an  Activity  System  and  the  Six  Clusters  of  Factors  That  Affect  the  Diffusion  of  the  Internet 


An  Activity  System 

Clusters  of  Factors  That  Affect  the  Diffusion  of  the  Internet 

Individual  or  subject 

User  characteristics  and  perceptions 

Norms  of  use,  conventions,  and  rules 

Cultural  and  organizational  issues,  norms  of  use,  legitimate  activities, 
"vision  of  learning" 

Tools  or  mediating  artifacts 

Tools,  design,  and  impersonal  supports 

Division  of  labor 

Social  issues  including  roles,  scaffolding,  mentoring,  communication 

Object  or  outcome  of  activity 

Individual  learning,  adoption,  conceptual  change 

Community 

Group  learning,  adoption,  conceptual  change 

Empirical  Base 

The  investigation  actually  began  in  the  fall  of  1994,  when  an  ad  hoc  group  of  graduate  students,  together 
with  their  advisor,  formed  the  Internet  Task  Force.  During  the  1994-95  academic  year,  the  group  attempted  to 
identify  facilitators  and  challenges  to  Internet  use  within  the  Division  of  Technology  and  Special  Services,  which 
housed  the  instructional  technology  master’s  and  doctoral  programs  (collectively  known  as  "I.T.”).  Later,  the 
population  under  surveillance  was  expanded  to  include  the  entire  School  of  Education.  The  purpose  of  the  study  was 
to  articulate  the  individual  conceptual  changes  and  group  processes  of  members  of  the  School  of  Education  as  they 
learn  the  basics  of  mediated  communication,  deal  with  their  concerns  and  learning  anxieties,  develop  expertise, 
adopt,  and  eventually  reaffirm  or  reject  the  use  of  the  Internet  to  support  teaching  and  learning. 

Building  on  a prior  study  of  Electronic  discussion  groups,  Using  e-mail  as  an  instructional  strategy  in  a 
graduate  seminar  by  Wilson,  Lowry,  Koneman,  and  Osman-Jouchoux  (1994),  members  of  the  Internet  Task  Force 
explored  two  facets  of  Internet  usage:  Affordances  and  constraints  of  the  Internet  for  learning  and  instruction 
(Ryder  & Wilson,  1996)  and  Cultural  assimilation  of  the  Internet  (Wilson,  Ryder,  McCahan,  & Sherry,  1996).  They 
also  studied  The  Dynamics  of  Collaborative  Design  (Sherry  & Myers,  1998)  and  documented  the  work  of  the 
Internet  Task  Force  (Sherry,  1996)  as  the  team  created  the  web  page  for  the  School,  using  the  university  e-mail 
system  and  the  WWW  to  share  information  and  negotiate  differences  of  opinion.  All  of  these  documents  are 
available  on  line. 

In  spring  1995,  one  member  interviewed  representative  members  of  the  faculty;  two  others  interviewed  a 
focus  group  of  students;  and  another  member  (Sherry,  1997)  analyzed  the  results  of  a survey  of  73  students,  faculty, 
and  staff  regarding  their  use  of  e-mail  and  the  Internet  for  instructional  purposes.  Five  factors  emerged  from  these 
various  studies:  (a)  clear  benefit  and  value,  (b)  self-efficacy,  (c)  finding  a voice  and  having  something  to  say  (also 
referred  to  as  mediated  writing  proficiency),  (d)  personal/cultural  compatibility,  and  (e)  proper  scaffolding. 

The  first  two  factors  to  emerge  from  these  earlier  studies,  namely  clear  benefit  and  value,  and  self-efficacy, 
were  identified  by  Bandura  (1982)  under  his  theory  of  self-efficacy  as  a mediator  of  performance  and  achievement. 
"Finding  a voice  and  having  something  to  say"  was  identified  by  Berge  (1997).  It  was  also  identified  and  explored 
by  Fishman  (1997)  as  written  communication  apprehension  and  by  Sherry  (1998)  as  mediated  writing  proficiency. 

Rogers  identified  the  fourth  factor,  personal/cultural  compatibility  (compatibility  between  technology  and 
people's  learning  styles,  self-concepts),  as  one  of  the  five  important  user  perceptions  of  an  innovation  - “the  degree 
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to  which  an  innovation  is  perceived  as  being  consistent  with  the  existing  values,  past  experiences,  and  needs  of 
potential  adopters".  In  this  study,  compatibility  includes  school  policies  and  norms  of  use  (Rogers,  1995:  15). 

The  fifth  factor  was  proper  scaffolding.  In  Hall  and  Hord’s  (1987)  Stages  of  Concern  model,  proper 
scaffolding  is  crucial  in  the  personal-concerns  stages  of  adoption  when  facilitators  should  visit  more  often  with 
potential  adopters  on  a face-to-face  basis  to  offer  assistance  and  encouragement.  Scaffolding  is  also  important  at  the 
management-concerns  stage  of  adoption  when  it  is  important  to  provide  "how  to  do  it"  workshops  that  address  the 
constantly  changing  task  issues  as  they  arise  (Hall  & Hord,  1987:  72). 

Two  years  later  (spring  1997),  the  current  study  began.  The  survey  was  repeated  with  the  same  instrument 
and  a stratified  sampling  of  278  students,  faculty,  and  staff  throughout  the  School  of  Education.  At  the  same  time,  a 
factor  analysis  was  performed  on  both  the  1995  and  1997  surveys  to  investigate  changes  in  trends  over  time.  Sherry 
(1998)  also  repeated  the  focus  group  and  interviews  in  spring  1998  using  a purposeful  selection  of  students,  faculty, 
and  staff,  comprising  early  adopters,  early/late  majority,  and  resisters  or  "laggards".  (See  Rogers,  1995.) 

Results 

In  1995,  participants  primarily  used  the  university  UNIX  system  for  their  e-mail  accounts,  except  for  the 
few  who  had  commercial  or  corporate  accounts.  By  1997,  the  variety  of  available  tools  had  increased.  86%  of  the 
respondents  used  e-mail;  74%  used  the  WWW;  and  60%  used  the  FirstClass  ™ BBS. 

Efficacy  x Value 

A factor  analysis  of  the  responses  to  the  survey  questions  that  dealt  with  reasons  for  using  the  Internet  (14 
items)  and  barriers/facilitators  to  using  the  Internet  (11  items)  was  performed  for  three  subsets  of  data:  (a)  1995 
responses  (I.T.),  (b)  1997  responses  (I.T.  cohort),  and  (c)  1997  responses  (Non-I.T.  cohort).  A principal  components 
analysis  with  Varimax  rotation  was  then  performed  on  the  three  sets  of  data.  This  revealed  the  general  trends  and 
changes  over  two  years  and  also  highlighted  the  differences  between  the  I.T.  and  the  non-I.T.  The  results  are 
presented  in  Tables  2 and  3. 

The  primary  reasons  for  Internet  use:  finding  information,  communicating  with  colleagues,  sharing 
information,  and  collaboration  (i.e.,  sharing  information  to  carry  out  an  intentional  activity)  varied  in  importance 
across  time  and  between  programs.  Note  the  emphasis  on  sharing  information  among  the  I.T.  cohort  vs.  the 
emphasis  on  finding  information  and  collaborating  among  the  Non-I.T.  cohort.  In  contrast,  the  facilitators  to 
Internet  use  were  remarkably  consistent  in  all  cases,  with  Bandura’s  efficacy  x value  accounting  for  about  half  the 
variance. 

Table  2.  Results  of  Factor  Analysis  on  Reasons  for  Use 


Survey 

1995:  I.T. 

1997:  I.T. 

1997:  Non-I.T. 

Factor  1 

Communicate  and  share 
information  (42%  of  variance) 

Share  information  (44%  of 
variance) 

Find  information  and  collaborate 
(50%  of  variance) 

Factor  2 

Find  information  (12%  of 
variance) 

Communicate  (1 1%  of 
variance) 

Share  information  (11%  of 
variance) 

Factor  3 

Collaborate  (9%  of  variance) 

Collaborate  (8%  of  variance) 

Communicate  (8%  of  variance) 

Factor  4 

Consult  with  advisor  (8%  of 
variance) 

Find  information  (7%  of 
variance) 

- 

Table  3.  Results  of  Factor  Analysis  on  Facilitators  to  Use 


Survey 

1995:  I.T. 

1997:  I.T. 

1997:  Non-I.T. 

Factor  1 

Clear  benefit  and  value  (33% 
of  variance) 

Clear  benefit  and  value  (32% 
of  variance) 

Clear  benefit  and  value  (38%  of 
variance) 

Factor  2 

Self-efficacy  (17%  of 
variance) 

Time  and  access  (15%  of 
variance) 

Self-efficacy  (16%  of  variance) 

Factor  3 

Mediated  writing  proficiency 
(10%  of  variance) 

Self-efficacy,  part  1 (10%  of 
variance) 

- 

Factor  4 

- 

Self-efficacy,  part  2 
(9%  of  variance) 

- 
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Personal/Cultural  Compatibility 

In  the  interviews,  students,  faculty,  and  staff  alike  reported  that  the  e-mail  feature  of  the  FirstClass  ™ BBS 
(i.e.,  CEO)  was  and  will  continue  to  be  an  efficient  means  of  communication  between  students  and  faculty.  The 
faculty  consensus  was  ’’it’s  better  than  voice  mail”,  and  students  remarked,  "I  get  better  response  from  professors  via 
e-mail  than  I do  by  phone  calls”.  Faculty  found  it  convenient  to  send  and  receive  papers  and  assignments  via  e-mail. 
Students  appreciated  the  more  rapid  turnaround  time  that  electronic  messaging  affords.  Moreover,  responses  from 
the  focus  group  indicated  that  there  was  social  pressure  among  some  of  the  learning  sub-communities  within  the 
school  that  reinforced  the  feeling  of  ”If  you  don’t  participate,  you  may  find  yourself  left  behind".  Both  the  interviews 
and  the  focus  group  presented  evidence  that  the  early  adopters  tended  to  be  intrinsically  motivated,  whereas  the  later 
adopters  often  felt  extrinsic  coercion. 

Besides  being  an  accepted  and  institutionalized  part  of  the  school’s  culture  as  a handy  communication  tool, 
some  faculty  members  predicted  that  using  electronic  conferencing  and  web  pages  on  CEO  for  distributed  learning 
might  prove  a viable  alternative  to  traveling  long  distances  to  provide  classes  for  geographically  dispersed  cohorts. 
At  the  time  the  interviews  were  conducted  (spring,  1998),  the  Administration,  Supervision,  and  Curriculum 
Development  (ASCD)  program  had  eleven  electronic  conferences  on  CEO.  Their  faculty  members  are  actively 
exploring  the  use  of  the  WWW  to  support  distance  and  distributed  learning.  A few  faculty  members  recently 
obtained  grants  to  design  on-line  courses. 

Not  all  faculty  and  students,  however,  are  early  adopters.  Early  adopters  often  expressed  a good  fit  between 
Internet  tools  and  their  personal  and  cultural  values.  Late  adopters  voiced  concerns  about  the  impact  of  the 
Internet  on  their  core  pedagogical  strategies,  indicating  that  it  may  not  support  their  vision  of  learning.  Since  internal 
resistance  to  innovations,  especially  the  use  of  interactive  technologies,  tends  to  change  the  traditional  role  of  the 
instructor  (Apple  Computer,  1995;  Yocam,  1998)  and  his/her  core  of  instructional  practice  (Elmore,  1996), 
institutionalization  of  Internet  and  WWW  tools  other  than  e-mail  will  most  likely  take  place  in  fits  and  starts.  A 
faculty  member  described  her  awareness  of  this  resistance: 

I'm  a little  bit  nervous  about  the  thoughtful  use  of  distributed  learning,  but  I do  see  that  the  first  and  most 
obvious  thing  is  reaching  those  audiences  that  are  not  otherwise  reachable  easily.  And  there's  a lot  of 
resistance  to  that.  I'm  in  big  trouble  now  because  I've  insisted  that  next  spring  when  I'm  supposed  to  go  to 
Durango  for  three  weekends  [an  eight-hour  drive  from  Denver],  an  equivalent  to  one  of  those  weekends  is 
going  to  be  distance  learning,  somehow.  They  absolutely  do  not  want  me  to  do  that! 

Nearly  all  interviewees  noted  that  there  was  no  incentive  system  in  place  for  them  to  adopt  Internet  and 
WWW  uses  other  than  e-mail.  Interview  comments  made  it  apparent  that  it  is  going  to  take  quite  a while  until 
distributed  learning  - a very  different  concept  from  traditional  forms  of  instruction  - becomes  part  of  the  school's 
culture. 

Finding  a Voice  and  Having  Something  to  Say 

In  an  activity  system,  the  "tool"  must  have  the  necessary  affordances  to  enable  the  actor  to  carry  out  an 
intentional  activity  - to  accomplish  an  intended  outcome.  If  the  tool  does  not  appear  to  the  user  to  be  the  most 
effective  means  of  accomplishing  his/her  objective,  whether  due  to  its  inherently  unfriendly  interface,  or  to  the  user's 
fear  of  writing  to  a public  audience  and  leaving  an  electronic  "paper  trail’’,  then  it  will  not  be  used.  In  this  case,  if  the 
Internet  "tools"  do  not  make  communication  easier  for  the  user,  then  all  the  training  in  the  world  will  not  turn  a 
resister  into  an  adopter. 

Students  and  faculty  who  are  just  beginning  to  experiment  with  electronic  conferencing  and  online 
communication  sometimes  exhibit  written  communication  and  lack  of  mediated  writing  proficiency.  Text -based 
communication  lack  social  cues,  and  therefore,  may  lead  to  misunderstandings  or  misinterpretation  of  the  author’s 
intent.  Moreover,  it  is  perceived  by  some  learners  to  be  more  reflective  than  spoken  interaction.  "The  very  act  of 
assembling  one's  thoughts  and  articulating  them  in  writing  for  a [computer]  conference  audience  appears  to  involve 
deeper  cognitive  processing"  (Berge,  1997:  10).  This,  in  turn,  may  lead  to  written  communication  apprehension. 

Fishman  (1997)  found  a significant  relationship  between  written  communication  apprehension  and  the  use 
of  Usenet  newsgroups  among  students  who  were  using  a combination  of  CMC  tools  in  a mediated  learning 
environment  called  the  "Collaboratory".  If  the  network's  interface  is  not  particularly  user-friendly  (as  in  the  older 
PINE  e-mail  interface  on  the  university  servers),  and  if  students  have  concerns  about  their  general  representational 
proficiency  (the  ability  to  represent  abstract  concepts  in  concrete  form)  or  their  mediating  writing  proficiency 
(writing  to  an  online  audience),  then  these  factors  could  potentially  affect  the  level  of  use  of  the  network. 

This  is  exactly  what  Wilson  and  his  colleagues  found  in  the  1995  interviews.  It  also  showed  up  as  the 
Factor  3 in  the  1995  factor  analysis  of  the  survey.  By  1997,  this  factor  had  disappeared  from  the  survey  results,  but 
was  still  flagged  by  an  interviewed  student.  She  alluded  to  new  forms  of  literacy  that  needed  to  be  explored  and 
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developed  - visual  literacy,  media  literacy,  and  literacy  in  terms  of  the  Internet  and  e-mail  - other  forms  of  literacy 
that  she  considered  appropriate  if  the  school  really  aspires  to  be  a community  of  learning.  A professor  also  discussed 
the  additional  cognitive  skills  that  are  necessary  when  dealing  with  on-line  text: 

It's  really  hard  for  me  to  read  on  a screen . / really  want  to  print  things  out  if  there's  very  much  of  it  and 
have  a hard  copy  in  my  hand.  What  we  don't  know  is  the  long-term  impact  of  making  those  kinds  of 
adjustments  to  the  learner.  For  example,  in  reading,  we  learn  how  to  read  stories  as  little  kids.  We  don't 
know  how  to  read  expository  text,  so  that  when  expository  text  gets  presented  to  us,  it's  new,  it's  different, 
it's  boring  We  don't  have  the  skills  unless  those  are  actively  taught  to  us  because  the  frameworks  that  we 
have  for  reading  are  about  stories. 

This  may  be  one  of  the  reasons  why  faculty  members  lagged  behind  students  in  publishing  on-line 
documents.  Three  faculty  and  eight  students  who  had  personal  web  pages  used  them  to  disseminate  full-text  versions 
of  their  publications.  Contributors  to  the  school's  "Scholarly  publications"  page,  which  espouses  a clearly  stated  set 
of  high  standards  for  publication,  consisted  of  nine  faculty  members  (three  of  whom  were  UCD  students  when  they 
first  carried  out  the  research  on  which  their  publications  were  based),  twenty  students,  and  seven  unaffiliated  second 
authors.  Another  explanatory  factor  might  be  faculty  members*  fear  of  compromising  intellectual  property  rights;  but 
ownership  and  copyright  issues  did  not  arise  in  any  of  the  interviews. 

A totally  different  aspect  of  "finding  a voice  and  having  something  to  say"  is  the  open  sharing  of  promising 
practices  among  faculty  who  have  used  the  Internet  and  the  WWW  effectively.  Some  faculty  members  are  using 
electronic  conferencing,  but  not  all  faculty  members  are  aware  of  the  possibilities  that  this  new  form  of  interactive 
communication  and  instruction  might  entail.  Faculty  meetings  generally  deal  with  administrative  issues,  leaving 
little  time  for  professors  to  share  innovative  instructional  strategies  that  have  proved  effective  with  their  students. 
This  is  beginning  to  change  with  the  introduction  of  workshops  for  honorarium  (part  time)  professors,  inaugurated 
by  the  University's  Office  of  Teaching  Effectiveness  (OTE);  and  with  the  creation  of  the  new  "Teaching  and 
Learning  with  Technology"  (TLT/LTTS)  mentoring  laboratory  for  faculty,  made  possible  by  a new  grant  to  the 
School  of  Education  (See  Grabinger,  paper  #199). 

Proper  Scaffolding 

This  refers  to  a support  structure  that  includes  a non-judgmental,  social  support  system,  one-on-one 
mentoring  relationships,  and  removal  of  technical  hurdles  to  the  innovation.  Existing  and  proposed  supports  were 
divided  into  two  categories:  impersonal  and  personal.  Impersonal  supports  comprised  brochures,  booklets,  on-line 
tutorials,  and  other  forms  of  print-based  or  electronic  performance  support  that  do  not  require  one-on-one  interaction 
with  a graduate  assistant,  fellow  student,  faculty  member,  or  staff  member.  Personal  supports  comprised  help  from 
graduate  assistants,  on-line  help  from  the  university's  network  services  staff,  direct  instruction  in  class,  and  free 
workshops  conducted  by  faculty,  staff,  technically  adept  students,  or  other  perceived  "experts". 

Interview  participants  were  asked  to  suggest  improvements  to  the  school's  support  structure.  Survey 
participants  were  asked  to  rank  a set  of  eight  supports  for  training  and  performance  using  e-mail  and  the  WWW, 
with  “1”  = most  useful  support  and  “8”  = least  useful  support.  The  eight  proposed  supports  were  formal  classes, 
brochures,  informative  booklets,  on-line  tutorials,  paper  tutorials,  interactive  computer  demonstrations,  individual 
assistance  by  graduate  students,  and  workshops.  Participants  who  rated,  rather  than  ranked,  the  suggested  supports, 
confounded  the  results  in  about  5%  of  the  cases. 

In  1995,  the  supports  ranked  highest  by  the  survey  participants  were  formal  classes  (ranked  #1  by  32%  of 
respondents)  and  workshops  (ranked  #2  by  30%  of  respondents).  Booklets  were  least  popular,  with  over  20%  of  the 
respondents  ranking  them  #6  or  #8.  Individual  attention  by  graduate  students  was  bipolar:  22%  of  respondents 
ranked  it  #8  and  1 7%  of  respondents  ranked  it  either  #1  or  #3.  This  was  worthy  of  further  investigation. 

There  were  significant  positive  Spearman  correlations  (p<.05)  between  the  self-efficacy  factor  and  the 
relative  rankings  of  both  formal  classes  (+.026)  and  individual  attention  by  graduate  students  (+.029).  There  was  a 
significant  negative  correlation  (p<.05)  between  the  self-efficacy  factor  and  the  relative  ranking  of  informative 
booklets  (-.025).  This  led  to  the  conclusion  that  respondents  who  were  low  in  self-efficacy  considered  personal 
supports  and  scaffolding  to  be  relatively  useful  as  compared  with  impersonal  supports  such  as  booklets. 

In  1997,  formal  classes  (ranked  #1  by  27%  of  respondents)  and  workshops  (ranked  #1  by  32%  of 
respondents)  topped  the  list  of  preferred  supports.  Booklets  (ranked  #8  by  24%  of  respondents)  were  still  least 
popular.  However,  the  bipolarity  in  the  ranking  for  individual  attention  by  graduate  assistants  had  disappeared,  with 
25%  of  respondents  ranking  this  personal  support  as  #1.  In  contrast,  on-line  tutorials  were  now  bipolar  (ranked  #2 
by  18%  and  #7  by  19%  of  respondents).  These  rankings  indicated  that  the  respondents  generally  preferred  personal 
scaffolding  to  impersonal  supports,  but  held  varying  opinions  concerning  the  specific  type  of  support  that  they 
would  like  to  see  implemented. 
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Faculty  and  students  alike  were  often  unaware  of  the  range  of  supports  that  already  existed.  For  example, 
the  university's  network  services  developed  an  extensive  set  of  free  workshops  - exactly  the  type  of  1-  or  2-hour 
workshops  that  students  said  they  would  like  to  see  offered  - but  students  were  not  aware  of  them.  Brochures, 
schedules,  and  job  aids  explaining  various  facets  of  the  university's  computers  and  directions  for  using  them  were 
freely  available  outside  the  network  services  office.  Copies  were  also  available  in  the  School  of  Education's 
computer  lab.  Again,  staff  and  graduate  assistants  were  not  aware  of  their  existence,  so  they  did  not  publicize  them 
to  new  users.  An  important  aspect  of  an  activity  system  - and  any  system,  for  that  matter  - is  the  fact  that  for  it  to 
function  efficiently,  information  must  flow  freely  throughout  the  system,  to  and  from  all  participants.  If  a support 
exists,  but  users  are  not  aware  of  it  (cf.  Rogers,  1995,  “observability”),  then  it  is  like  not  being  there  at  all! 

Recommendations 

This  study  resulted  in  a set  of  recommendations  for  improving  access,  functionality,  training,  and  technical 
support;  use  of  communication  channels  to  convey  important  information  about  existing  support  and  scaffolding 
structures;  and  the  use  of  electronic  conferences  to  enhance  and  enrich  classroom  discussions.  It  also  highlighted  the 
need  for  an  incentive  system,  and  for  ways  that  faculty  might  share  promising  practices  that  use  the  Internet  to 
support  instruction. 

Based  on  the  results  of  this  study,  recommendations  for  future  investigation  and  development  are  listed  in 
Table  4.  Other  colleges  and  universities  that  are  considering  infusing  instructional  technology  into  their  general 
education  programs  (such  as  PT3  grantees)  might  find  these  recommendations  applicable,  as  well. 

Table  4.  Recommendations 


Have  better  publicity  about  existing  aids  and  supports,  using  multiple  channels  of  communication. 

Have  better  communication  and  collaboration  between  the  School  of  Education,  the  university,  and  the  university 

network  services,  possibly  sharing  duties  where  they  overlap. 

Develop  a flexible  schedule  of  Internet  demonstrations  or  open  lab  workshops  with  optional  student  attendance. 
Hire  more  graduate  students  in  the  School  of  Education  computer  lab  who  have  the  skills  and  the  time  to  help 

individual  students  with  specific  problems. 

Create  a permanent  position  for  an  in-building  technical  support  person  who  will  be  available  in  person  or  by 

telephone  when  classes  are  in  session. 

Consider  the  possibility  of  developing  on-line  tutorials  for  commonly  used  Internet  tools. 

Encourage  "show  and  tell"  sessions  among  faculty  members  to  discuss  and  share  ideas,  strategies,  and  promising 

practices  for  Internet  use  beyond  simple  e-mail  messaging  to  support  teaching  and  learning. 

Encourage  students  to  create  on-line  research  management  products  and  portfolios  to  serve  as  models  of  scholarly 
products  for  new  students,  and  to  elicit  feedback  from  peers,  colleagues,  and  experts. 


Implications  for  Institutionalization 

Based  on  this  study's  findings,  the  use  of  e-mail  will  continue  to  increase  because  CEO  has  become  a 
commonly  accepted  mediating  artifact  within  the  norms  and  conventions  of  the  school.  All  interviewees  and  focus 
group  members  used  CEO  regularly  in  1997,  in  contrast  with  the  more  sporadic  e-mail  use  in  1995.  As  members  of 
a commuter  campus,  students  stated  that  it  is  more  convenient  to  connect  from  home  or  work  on  a regular  basis 
rather  than  to  travel  to  campus  and  pay  for  parking  (which  may  not  even  be  available!) 

Use  of  the  WWW  is  increasing,  and  will  continue  to  increase,  now  that  the  foremost  item  on  the  students' 
wish  list  - free  ISP  service  through  student  fees  to  the  university  - has  been  granted.  Students  who  primarily  used 
CEO  were  beginning  to  discover  that,  contrary  to  the  WWW,  a FirstClass  ™ BBS  does  not  automatically  provide 
the  types  of  Internet-wide  search  engines  and  database  tools  the  they  would  like.  They  stated  that  they  would  like  to 
increase  their  access  to  research-based  databases,  on-line  libraries,  collections  of  legal  and  medical  information  — 
resources  that  are  freely  available  on  the  WWW  and  that  matched  their  own  educational  goals.  More  importantly, 
there  has  been  a recent  influx  of  out-of-state  applicants  to  the  School  of  Education,  who  found  out  about  the  school's 
programs  via  the  Web  page  created  by  the  Internet  Task  Force,  and  who  are  already  Intemet-enculturated. 

As  a result  of  this  study,  both  the  School  of  Education  and  the  university  have  become  more  aware  that  no 
electronic  helpdesk  can  offer  the  moral  support  that  new  users  need  as  they  deal  with  their  personal  and  task 
management  concerns.  There  were  three  notable  follow-ons  to  this  study: 

• A TLT/LTTS  lab  was  established  through  a FPSE  grant.  It  exists  primarily  for  the  purpose  of  mentoring 
faculty  in  Internet  use  and  other  forms  of  educational  technology  (See  Grabinger,  paper  #199). 
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• The  university’s  Office  of  Teaching  Effectiveness  (OTE)  runs  a "Boot  Camp  for  Professors"  each  summer. 
This  is  beginning  to  gain  in  popularity  and  has  been  particularly  successful  because  it  does  not  interfere  with 
teaching  activities  during  the  normal  academic  year. 

• Based  on  requests  for  one-on-one  scaffolding  by  faculty  as  well  as  students,  the  OTE  inaugurated  a new  work- 
study  program  this  summer.  A dozen  incoming  freshmen  in  September  1999  were  paid  to  take  a July  1999 
training  session  known  as  "Student  Instructional  Technology  Corps"  (SITC),  where  they  received  instruction 
that  enabled  them  to  serve  as  I.T.  assistants  to  various  academic  divisions  throughout  the  university.  However, 
the  SITC  eventually  ran  into  insurmountable  barriers  involving  early  registration  and  lack  of  funds  for 
freshman  work-study  students. 

Increased  usage  will  not  come  without  unintended  side  effects.  What  is  working  now  may  not  work  in  the 
future.  Students  with  slow  modems  and  insufficient  RAM  are  beginning  to  feel  a sense  of  frustration  with  their  older 
hardware  platforms.  As  new  users  come  on  board,  especially  those  who  are  unfamiliar  with  computer-mediated 
communication  (CMC),  they  will  continue  to  have  problems  with  modem  settings  and  other  software  issues. 
Moreover,  as  more  and  more  students  begin  to  use  their  free  student  accounts  for  PPP  access  to  the  WWW,  system 
capacity  will  be  stressed,  leading  to  carrier  drops  and  busy  signals.  All  of  this  can  be  frustrating  for  new  users.  Other 
educational  organizations  (Sherry,  Lawyer-Brook,  & Black,  1997)  have  had  to  increase  their  server  capacity  or 
purchase  additional  servers  to  keep  up  with  increased  on-line  traffic  as  new  users  begin  to  use  the  system  on  a 
regular  basis.  Any  instructional  technology  program  or  educational  institution  will  eventually  have  to  deal  with  this 
issue. 

Most  importantly,  linear  adoption  models  such  as  Rogers’  (1995)  Diffusion  of  Innovations  and  Hall  and 
Hord’s  (1997)  CBAM  model  (in  which  time,  rather  than  personal/cultural  compatibility,  separates  early  from  late 
adopters)  may  not  apply  to  learning  organizations  such  as  graduate  schools.  Often,  it  is  the  late  adopters  who  have 
insights  into  unintended  side  effects  of  technological  innovations.  Moreover,  expanded  conceptual  frameworks  that 
are  based  on  activity  theory,  systems  theory,  or  complexity  theory  (see  Wilson,  Sherry,  Dobrovolny,  Batty,  & Ryder, 
in  press)  may  provide  one  with  deeper  understandings  into  how  individual  and  organizational  learning  and  change 
occur  over  time  within  institutions  of  higher  education. 
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Abstract 

Library  service  is  one  of  the  critical  elements  for  designing  a successful  distance  program  in  terms  of 
learner  support  at  a distance . Important  services  of  the  library  model  for  the  Instructional  Systems  Technology  (1ST) 
online  program  at  Indiana  University  (IU)  were  identified,  based  on  the  needs  of  students,  faculty,  librarians , and 
administrators.  Interviews  and  survey  were  major  methods  used  in  this  study. 

Introduction 

Since  higher  education  involves  students  in  inquiry,  research,  and  critical  thinking,  library  accessibility  and 
support  play  an  important  role  in  the  success  of  distance  education.  According  to  Lebowitz  (1997),  library  services 
to  distant  students  include  instruction  in  the  use  of  libraries  and  library  resources,  contact  information  for  requesting 
assistance  from  the  library,  information  about  requesting  materials  not  available  at  local  site  libraries,  quick 
turnaround  time  for  materials  requested  from  the  library  collection,  quick  response  to  requests  for  reference 
assistance  and  guidance,  and  quick  response  time  to  non-library  related  questions.  Usually,  the  existing  library 
services  in  traditional  universities  do  not  adequately  meet  the  special  needs  of  distance  learning.  Without  equitable 
access  to  library  resources  and  librarians,  students  at  a distance  cannot  perform  the  same  research  functions  as  on- 
campus  students,  reducing  their  academic  experience  and  creating  frustration.  Such  stressful  experience  could  be 
among  the  factors  that  cause  a high  dropout  rate  in  distance  education.  As  a result,  many  principles,  guidelines,  or 
services  have  been  developed  to  ensure  quality  library  service  for  the  extraordinary  growth  of  technology-mediated 
distance  learning  in  higher  education  (Cooper,  1998). 

To  develop  a quality  library  service  for  distance  education,  the  users,  the  instructors  and  off-campus 
students,  and  the  service  providers,  the  librarians  and  administrators,  should  be  involved.  “Regardless  of  which 
model  is  developed  and  what  variation  used,  it  is  imperative  that  institutions  and  librarians  recognize  that  distance 
education/off-campus  students  are  entitled  to  library  services  which  are  comparable  to  those  available  to  on-campus 
students  (Lebowitz,  1997).”  To  make  sure  that  students  at  a distance  have  equal  access  to  library  resources,  a 
program  designer  needs  to  better  understand  these  students  and  try  to  meet  their  needs  and  expectations.  In  addition 
to  students,  a designer  must  consider  and  support  the  faculty,  who  are  the  key  to  making  all  materials  alive. 
Moreover,  developing  effective  library  services  for  distant  students  is  not  just  a library  issue;  it  can  never  be  tackled 
without  the  support  of  an  institution’s  administrators.  Therefore,  before  a quality  library  service  can  be  developed, 
three  questions  need  to  be  answered.  First,  what  are  the  expectations  and  needs  of  students  and  faculty  in  online 
programs?  Second,  what  kinds  of  library  services  are  critical  to  faculty  and  students  in  online  programs?  Finally, 
are  these  expectations  and  needs  met  by  the  services  offered? 

The  purpose  of  this  study  was  to  identify  important  library  services  for  the  1ST  distance  program,  based  on 
the  needs  and  expectation  of  students,  faculty,  librarians,  and  administrators.  The  important  library  services  for 
distance  education  were  identified  through  a review  of  the  literature,  interviews  with  students,  faculty,  librarians  and 
administrators,  researchers’  previous  experiences,  brainstorming,  and  discussion  with  librarians. 

Library  Services  for  Distance  Education 

A literature  review  was  conducted  to  compile  a list  of  library  services  for  distance  education  as  a 
framework  of  this  study.  Guidelines  for  distance  learning  library  services  developed  in  1998  by  the  Association  of 
College  and  Research  Libraries  (ACRL),  the  nation’s  largest  professional  association  for  academic  libraries,  was 
reviewed,  along  with  others  suggested  in  various  articles  and  publications  (Brophy,  1997;  Butler,  1997;  Derlin  & 
Edward,  1997;  Jones,  1997;  Kirk  & Andrea,  1999;  MacDougall,  1998;  Niemi  & Barbara,  1998;  Tolsma,  1997).  The 
authors  also  brainstormed  in  order  to  identify  potential  library  services  based  on  past  experiences  and  observations. 
Considerable  overlapping  among  services  derived  from  these  various  sources  were  found  and  eliminated.  Finally, 
library  services  for  distance  education  were  grouped  into  the  following  six  categories: 
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1.  Access  to  research  materials.  The  services  in  this  category  include  online  databases,  online  library 
catalogs,  physical  library  collections  (both  at  IU  and  on  local  sites),  and  reserve  readings. 

2.  Delivery  of  requested  materials.  This  category  includes  services  for  delivering  requested  materials  from 
the  library  to  a student  via  postal  service,  FAX,  or  various  online  delivery  options. 

3.  Instruction.  In  order  to  identify  and  utilize  appropriate  library  research  resources  and  materials,  distant 
students  need  librarians’  instruction  detailing  those  resources  and  procedures.  Services  in  this  category 
include  providing  instruction  as  part  of  a program  orientation,  face-to-face  instruction,  instruction  by 
phone,  online  tutorials,  and  online  asynchronous  and  synchronous  instruction. 

4.  Help.  While  instruction  covers  more  pre-formatted  techniques  and  formal  teaching  or  curriculum  such  as 
tutorials,  spontaneous  or  specified  questions  are  answered  by  “Help”  services,  which  may  take  the  form  of 
lists  to  browse,  FAQ’s,  or  possibly  a Knowledge  Base.  The  services  in  this  category  related  to  assistance 
in  identifying  subject  materials  that  students  may  need  through  the  program,  highlighting  indexes  and  1ST 
core  journals,  standard  reference  books  within  a discipline,  assistance  with  reference  formatting,  search 
help  specific  to  a discipline,  and  identifying  and  accessing  local  library  resources. 

5.  Policies  and  procedures.  Available  library  services,  as  well  as  services  not  provided,  should  be  clearly 
defined  in  policies  and  procedures.  Then  distant  students  will  know  the  availability  of  certain  library 
services,  the  best  way  to  use  these  services,  and  the  correct  channels  of  communication.  This  category 
includes  handling  requests  from  individuals  outside  the  primary  audience,  referring  requests  that  the 
library  cannot  handle  to  the  most  appropriate  IU  or  other  resource,  limits  on  services,  and  any  costs  that 
students  may  incur. 

6.  Faculty  assistance.  Developing  relationships  and  collaborative  efforts  with  faculty  will  be  a critical  factor 
in  the  success  of  library  services  for  distance  education.  This  category  includes  providing  faculty 
assistance  on  copyright  issues  and  procedures,  reserving  materials,  developing  library  instructional 
components,  and  updating  news-and-notes  type  of  current  awareness  publications. 

Methods 

Fourteen  subjects  were  initially  identified  as  interviewees  based  on  their  substantial  experience,  potential 
personal  interests,  or  their  involvement  in  distance  education  or  the  1ST  distance  program.  Two  of  these  initial 
candidates,  the  Associate  Vice  President  for  Distributed  Education  and  the  Dean  of  University  Libraries,  were  not 
interviewed  due  to  time  and  scheduling  constraints.  However,  administrative  perspectives  were  still  sustained  by 
interviewing  the  Head  of  the  IU  Undergraduate  Library.  While  there  were  minor  variations  in  the  questions  asked  of 
each  interviewee,  they  typically  centered  on  the  following  topics:  1)  What  do  you  think  of  when  someone  mentions 
a phrase  like  ‘library  services  for  distance/distributed  education’?  2)  For  what  type  of  assignments  or  information 
do  you/your  students/library  users  use  the  library?  3)  How  quickly  should  students  expect  a response  or  delivery  of 
material  from  library  services? 

In  addition  to  the  interview  questions,  each  interviewee  completed  a Likert-type  scale  to  rate  the 
importance  of  specific  library  services,  as  they  would  be  applied  to  the  online  1ST  program.  This  survey  instrument 
was  designed  to  determine  how  important  the  services  are  to  the  institution’s  faculty,  administrators,  librarians,  and 
students. 

Results 

From  February  through  mid-April  of  2000,  the  researchers  interviewed  each  of  participants.  A list  of  open- 
ended  questions  was  initially  probed  which  led  to  free-thought  discussion  and,  sometimes,  brainstorming.  The 
interviews  themselves  lasted  anywhere  from  25  minutes  to  nearly  2 hours.  Clearly,  some  individuals  had  much  to 
say  and  sparked  some  creative  ideas.  The  survey  and  interview  questions  were  also  distributed  via  e-mail  to  two 
students  who  were  not  able  to  physically  participate  in  the  interview  process  because  of  distance  constraints.  In  all, 
three  faculty  members,  five  librarians,  one  1ST  distance  education  committee  member,  and  four  students  were 
interviewed  and/or  completed  a survey,  for  a total  of  thirteen  participants.  Only  twelve  of  these  were  given  the  final 
official  survey. 

As  to  open-ended  questions,  different  interviewees  repeatedly  mentioned  a number  of  issues.  Since  these 
comments  were  unsolicited  and  unprompted  (coming  before  the  survey  listing  specific  services),  these  indicated 
commonly  perceived  essential  needs. 

Database  access  was  one  of  the  most  frequently  identified  services.  On  several  occasions,  the  ERIC 
database  was  specifically  mentioned.  Along  with  the  necessity  for  students  to  access  research  databases,  the  issue  of 
accessibility  for  students  beyond  the  IU  network  domain  was  a concern.  Traditionally,  the  databases  purchased  by 
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libraries  have  only  been  accessible  on  campus,  while  some  are  more  strictly,  available  only  inside  libraries.  Very 
recently  the  libraries  have  reached  agreements  with  a number  of  database  vendors  to  allow  access  to  IU  students  not 
physically  shown  on  campus. 

Closely  related  to  database/literature  indexing  and  abstracting  access,  availability  of  full-text  databases  was 
repeatedly  identified  as  a critical  service.  Several  databases  offered  now  through  IU  libraries  do  contain  full- 
text/full -image  inclusion  linked  to  the  indexing.  The  libraries  also  offer  a handy  utility  for  identifying  specific 
journals  that  may  contain  full-text  entries.  An  alternative  method  of  offering  full-text  articles  may  be  accomplished 
through  scanning  the  materials  held  by  the  libraries.  While  providing  the  full-text  of  research  items  is  highly 
important  to  student  research,  the  hit-and-miss  coverage  currently  available  may  create  problems  in  distance 
learning.  For  instance,  if  students  are  only  using  items  they  can  immediately  obtain  in  full-text  and  forgoing  the 
lengthy  process  of  obtaining  potentially  more  relevant  and  academic  items  only  physically  available  in  libraries,  is 
the  academic  quality  of  their  research  suffering? 

Reserve  readings  also  surfaced  repeatedly  in  discussions.  Both  students  and  faculty  considered  electronic 
reserves  to  be  an  essential  library  service  for  distance  education.  While  the  Indiana  University-Purdue  University  at 
Indianapolis  (IUPUI)  Library  has  offered  electronic  reserves  for  several  years,  the  IUB  libraries  have  not  yet 
initiated  such  a service.  Reportedly,  the  libraries  are  exploring  this  with  a hoped-for  availability  by  August  2000. 

Document  delivery  was  another  commonly  cited  necessary  service.  Whether  through  US  postal  delivery, 
fax,  or  online  delivery  options,  interviewees  recognized  the  necessity  of  having  books,  articles,  or  other  library 
materials  delivered  upon  request.  It  is  still  unclear  exactly  what  services  would  be  available  to  1ST  distant  students. 
While  the  Library  Distributed  Education  Services  Web  pages  indicated  that  books  and  articles  from  the  IU  Libraries 
could  be  checked  out  and  delivered  to  distant  students  free  of  charge  (http://www.indiana.edu/~libdist/des.htm),  the 
Distributed  Education  Librarian  and  Head  of  the  Undergraduate  Library  contradictorily  pointed  out  that  books  held 
by  the  IU  libraries  would  not  be  delivered  to  off-campus  students.  IU  libraries  similarly  would  not  loan  books  from 
other  libraries  for  off-campus  students;  however,  they  would  interlibrary  loan  articles  from  journals  the  IU  libraries 
would  not  subscribe  to  for  off-campus  students. 

Another  theme  repeatedly  expressed  by  interviewees,  and  previously  mentioned  as  prominent  in  the 
literature  related  to  library  services  for  distributed  education,  was  that  distant  students  should  have  access  to  library 
resources  and  services  comparable  to  what  students  with  physical  access  to  the  libraries  would  receive.  An 
interesting  side  note,  faculty  and  students  mentioned  this  more  often  than  did  the  librarians  who  were  interviewed. 

Access  to  librarians  or  library  staff  was  another  repeated  theme.  The  reasons  for  contacting  library  staff 
may  include  requesting  research  assistance,  obtaining  a quick  reference  answer,  initiating  or  asking  about  services, 
or  even  asking  about  other  IU  divisions  or  services  beyond  the  library. 

Closely  related  to  the  above  issue  regarding  access,  several  librarians  interviewed  mentioned  the  necessity 
of  providing  a “hand-holding”  service  to  users:  Depending  on  additional  student  support  services  offered  by  IU,  the 
library  could  well  become  the  “friendly  voice”  that  students  seek  out  whenever  they  have  questions  and  do  not  know 
where  else  to  turn.  Students  will  also  need  a high  degree  of  personal  assistance  and  attention  as  they  struggle  to  use 
online  databases  or  card  catalogs,  explore  where  they  should  look  for  different  types  of  information,  need  advice  on 
research  topics,  etc. 

The  final  major  finding  from  the  open-ended  questions  focuses  on  the  idea  of  responsiveness.  Interviewees 
were  asked  a two-part  question:  (1)  “How  long  should  students  have  to  wait  for  a response  to  an  initial  query?”  and 
(2)  “If  the  library  will  deliver  materials  directly  to  students,  how  quickly  should  the  students  expect  to  get  those 
materials?”  The  answers  showed  an  enormous  range,  from  immediately  to  72  hours  for  the  first  question  and  from 
immediately  (if  electronic)  to  two  weeks  for  the  second  question.  Popular  answers  for  the  first  part  were  either 
immediate  acknowledgement  of  receipt  or  responses  within  24  hours.  Popular  answers  for  the  second  part  were 
either  delivery  in  two  to  three  days  or  within  five  days. 

In  particular,  It  is  necessary  to  point  out  the  disparity  between  user  expectations  and  the  expectations  of 
libraries.  While  answers  from  faculty  and  students  generally  indicated  a shorter  response  time,  the  answers  given  by 
librarians  tended  toward  the  longer  time  frames.  Distant  students  would  particularly  have  a need  for  quick  access  to 
library  materials  and  librarians’  assistance.  If  the  library  services  provided  were  not  viewed  as  responsive,  the 
perception  of  the  service  as  a whole  would  have  been  degraded. 

At  the  end  of  each  interview,  participants  were  asked  to  fill  out  a survey  to  rank  their  perception  of  the 
necessity  of  several  services.  The  results  have  been  tabulated  and  are  attached  as  Table  1.  Answers  were  given  on  a 
scale  of  one  to  five,  one  being  of  little  relevance  and  five  being  highly  relevant/critical.  The  averaged  results  speak 
for  themselves,  but  here  are  some  highlights  from  these  findings. 
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Table  1 


Mean 

Access  To  Research  Materials 

- Online  Database  4.92 

- Online  Card  Catalogs  4.42 

- Physical  Library  Collection  4.08 

- Materials  From  Other  Libraries  3.75 

Delivery  of  Requested  Materials 

- Postal  Service  4.08 

- Fax  3.50 

- Online  Delivery  Options  4.67 

Instruction 

- Part  of  Program  Orientation  4.33 

- Face-to-Face  Instruction  2.36 

- Online  Tutorials  4.42 

- Online  Asynchronous  Help  4.42 

- Online  Synchronous  Help  3.25 

- Phone  Help  4.75 

Help 

- Highlighting  Indexes  and  Core  Journals  3.58 

- Standard  Reference  Books  w/in  Discipline  3.83 

- Search  Help  specific  to  Discipline  3.92 

- Assistance  w/ Reference  Formatting  3.17 

- Identifying  and  Accessing  Local  Library  Resources  4.00 

Policies  and  Procedures 

- Handling  requests  from  students/others  outside  program  4.33 

- Handling  requests  & referrals  for  services  not  provided  3.82 

- Limits  on  Services  Provided  4.67 

- Pass-back  costs  4.75 

Faculty  Assistance 

- Copyright  Issues  and  Procedures  4.50 

- Reserve  Materials:  Policies  & Procedures  4.50 

- Developing  Library  Instructional  Components  4.25 

- Library  Update/News  & Notes  publication 3.67 


Online  database  access  (mean  = 4.92)  was  regarded  as  highly  critical.  When  options  were  available  for 
delivery  of  materials,  most  preferred  the  ability  to  send  or  receive  these  through  online  venues  (4.67).  Several 
instructional  methods  were  desirable  for  librarians  to  teach  and  to  guide  users  in  resources  and  procedures,  but  most 
respondents  agreed  that  phone  contact  (4.75)  was  a necessary  service  to  provide.  While  the  policy  and  procedure 
questions  focused  more  on  helping  library  staff,  the  overall  response  indicated  that  users  should  be  made  aware  of 
the  limits  (4.67)  and  pass-back  costs  (4.75)  for  services  offered.  Assisting  faculty  with  policies  and  procedures 
regarding  reserve  readings  (4.50)  was  rated  highly.  Several  interviewees  also  looked  to  library  services  to  provide 
guidance  on  matters  of  copyright  (4.50). 

Conclusions  and  Recommendations 

This  study  tried  to  identify  important  library  services  for  successful  distance  education.  From  the  open- 
ended  questions,  different  interviewees  repeatedly  offered  a number  of  concerns.  Since  these  comments  were 
unsolicited  and  unprompted,  these  indicate  commonly  perceived  essential  needs.  As  previously  stated,  several 
library  services  deemed  important  or  critical  for  distant  students  were  identified  through  the  literature  review  and 
brainstorming.  Most  notable  of  these  themes  was  the  overarching  philosophy  that  library  services  for  distant 
students  should  be  comparable  or  equitable  to  the  services  provided  to  on-campus  students  (ACRL  Guidelines, 
1997;  Lebowitz,  1997).  As  the  Chair  of  the  Middle  States  Accreditation  Association  stated,  “there  can  be  no  real 
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differences  in  the  quality  of  library  support  on  or  off  campus.  If  the  same  level  of  quality  is  to  be  maintained, 
comparable  - not  necessarily  the  same  - library  resources  and  services  are  imperative  (Lebowitz,  1997).” 

User  Expectations 

A second,  and  often  overlooked,  theme  emerged  that  may  prove  critical  to  the  continued  success  of  library 
services  for  distance  education,  namely  customer  satisfaction  based  on  user  expectations.  Libraries  perceptions  of 
user  expectations  often  do  not  coincide  with  what  the  users  themselves  view  as  important.  Based  on  a 1 995  report 
by  Edwards  and  Bourne,  librarians  “tend  to  emphasize  empathy,  tangibles,  and  customer/staff  relationships.  Yet, 
academic  library  users  tend  to  attach  greater  importance  to  reliability  and  responsiveness  (Cooper  & Dempsey, 
1998).”  In  order  to  be  “successful”  by  user  standards,  library  services  will  need  to  fully  understand  their  users  and 
user  expectations  and  to  exceed,  not  just  meet,  those  expectations. 

Access 

Access  to  research  resources  (databases  and  library  catalogs)  and  to  research  materials  is  of  primary 
importance  in  the  process  of  academic  inquiry.  Several  resources  are  currently  available.  Several  of  these  will 
improve  in  usability.  More  will  undoubtedly  follow.  What  is  important  at  this  stage  is  for  library  services  to 
actively  market  these  resources.  Students  and  faculty  need  to  be  made  aware  of  the  best  and  most  appropriate  tools 
available  to  them.  Faculties  in  turn  need  to  make  students  aware  of  these  resources  and  encourage  their  use  for  class 
assignments. 

Delivery  of  materials 

Information  obtained  on  what  materials  are  available  to  distant  students  and  how  they  can  be  delivered  is 
conflicting  and  somewhat  confusing.  Clear  policies  and  procedures  need  to  be  established  and  disseminated  so 
students  will  know  what  they  are  entitled  to,  what  options  are  available  for  getting  these  materials,  and  how  quickly 
they  can  expect  them. 

Instruction 

Students  will  continue  to  seek  assistance  from  librarians  at  the  last  minute.  Flexible  options  for  interaction 
and  just-in-time  instruction  will  need  to  be  offered,  particularly  through  phone  contact  and  online  options.  However, 
students  need  to  be  made  aware  of  the  library  services  available  to  them,  who  to  contact,  and  how  to  use  library 
resources.  The  most  effective  way  of  exposing  1ST  online  students  to  the  library  service  available  to  them  is  by 
inviting  the  Distributed  Education  librarian  to  give  a presentation  during  their  initial  orientation  sessions  in  order  to 
introduce  the  service,  explain  the  service  available,  and  demonstrate  some  of  the  tools  they  will  use  ffo  research. 
With  the  intermediated  web  page  to  statement  what  service  are  available  and  how  to  access  to  the  service,  students 
can  be  remaindered  what  service  are  available,  where  and  how  to  access  later  on  during  the  semester. 

Help 

Several  tutorials  and  guides  are  currently  available  on  the  IU  libraries’  Web  pages.  A searchable  list  of 
Frequently  Asked  Questions  or  equivalent  Knowledge  Base  would  also  prove  helpful.  These  may  help  students  in 
getting  connected  to  databases  and  started  in  their  searches.  However,  these  are  geared  to  a more  generic  audience 
and  may  not  answer  specific  questions  1ST  students  may  have.  The  Distributed  Education  librarian  has  offered  to 
develop  specific  help  pages  that  can  be  geared  to  individual  courses  and  linked  from  that  course’s  Web  pages. 
Given  the  current  staffing  shortage  in  her  department,  this  is  not  a long-term  option.  A better  alternative  for  1ST 
students  would  be  an  intermediate  Web  site  that  offers  advice  for  library  research  and  appropriate  resources. 

Policies  & Procedures 

In  order  to  avoid  confusion  and  ambiguity,  policies  and  procedures  outlining  the  services  provided, 
including  limits  on  service,  procedures,  and  any  costs  involved  need  to  be  established  and  made  widely  available. 
Faculty  Assistance 

The  Distributed  Education  librarian  has  already  begun  contacting  departments  and  divisions  offering 
distance  education  courses  to  develop  rapport  and  advertise  library  services  for  distance  learning.  Librarians  should 
also  offer  assistance  to  faculty  developing  online  courses  in  order  to  determine  the  potential  information  and 
research  needs  of  students  in  these  courses.  If  any  courses  have  an  extensive  research  component,  the  librarians 
could  be  invited  to  serve  as  a “visiting  faculty”  resource  sitting  in  for  online  courses  and  offering  instruction  or 
individual  assistance  to  students.  Reserves  need  to  be  addressed,  and  quickly.  The  demand  is  apparent. 

Staffing 

The  current  staffing  of  Distributed  Education  Library  Services  is  not  sufficient.  Ashley  Vollmer  currently 
works  10-12  hours  serving  all  IUB  distance  education  students.  If  any  significant  demand  starts  to  flow  in,  she  will 
not  be  able  to  fulfill  all  requests  in  a timely  fashion.  A full-time  staff  position  is  required  to  adequately  research, 
plan,  and  build  library  services.  Even  though  the  ACRL  Guidelines  and  other  sources  recommend  that  library 
services  for  distance  education  should  be  funded  by  the  institution  in  addition  to  existing  library  services  and  not 
from  existing  library  funds,  it  appears  that  the  latter  is  the  case  at  IU.  The  IU  library  personnel  budget  has  also 
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decreased  over  the  past  several  years,  making  it  necessary  for  the  library  to  do  more  with  less  staff.  External  funds 
need  to  be  appropriated  to  provide  sufficient  staff  to  serve  DE  students. 

Implications 

Based  on  all  the  evidence  found  in  this  study,  several  conclusions  and  recommendations  were  drawn.  It 
should  be  noted  that,  while  this  document  seeks  to  identify  the  library  services  needed  to  successfully  serve  the  1ST 
online  program,  the  adoption  or  implementation  of  many  of  these  conclusions  would  need  to  happen  outside  of  1ST. 
Some  of  these  recommendations  may  be  seen  as  impractical  based  on  varying  circumstances  endemic  to  other 
offices,  nonetheless  we  offer  them  based  on  user  perceptions,  literature  recommendations,  and  personal 
observations. 

Understanding  user  needs/expectations  is  critical  if  library  services  are  to  be  considered  successful  by 
users.  Without  a firm  understanding  of  the  unique  needs,  constraints,  and  idiosyncrasies  of  distant  students. 
Distributed  Education  Library  Services  will  not  be  able  to  target  or  otherwise  influence  user  expectations.  To 
achieve  this  goal,  libraries  and  the  1ST  department  should  conduct  similar  surveys  on  a regular  basis.  Not  only  will 
this  provide  feedback  on  the  performance  of  library  services  and  student  expectations,  but  it  will  also  inform 
students  of  the  services  available. 

The  results  of  this  study  can  assist  policymakers,  such  as  chief  academic  officers,  head  librarians,  and 
distance  program  committees,  as  well  as  faculty  and  students,  in  making  reasonable  and  informed  judgments  with 
regard  to  the  quality  of  learner  support  for  distance  education. 

The  results  obtained  from  interviews  and  document  analysis  will  help  to  guide  the  decisions  for  important 
or  critical  services  and  resources  to  be  included  in  the  model.  The  next  section  will  discuss  the  methodology  that 
was  used  to  investigate  the  research  questions  and  to  obtain  the  information  that  is  the  basis  for  identifying 
important  services  and  the  potential  needs  that  should  be  built  into  the  library  model. 
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Abstract 

With  the  current  influx  of  grants  and  national  technology  standards,  many  in  Instructional  Technology 
departments  that  serve  teacher  education  programs  wrestle  once  again  with  working  outside  initial  certification 
areas.  How  can  Instructional  Technology  departments  work  with  teacher  education  faculty  and  programs  to  ensure 
that  novice  teachers  will  be  able  to  meet  new  standards?  Relationships  of  an  increasing  cooperative  nature  are 
called  for.  This  paper  describes  a case  of  IT  and  teacher  education  unit  cooperation  and  its  results. 

Introduction 

With  the  current  influx  of  Preparing  Tomorrow’s  Teachers  To  Use  Technology  grants,  many  in 
Instructional  Technology  departments  who  serve  teacher  education  programs  wrestle  once  again  with  working 
outside  the  initial  certification  areas.  In  light  of  increasing  standards  and  accountability  in  teacher  education 
programs,  not  only  in  technology  but  also  in  all  content  areas,  how  can  Instructional  Technology  departments  work 
with  teacher  education  faculty  and  programs  to  ensure  that  novice  teachers  will  be  able  to  meet  these  standards?  The 
technology  standards  are  high  and  require  significant  attention  in  order  for  new  teachers  to  meet  these  standards, 
probably  more  attention  than  most  teacher  education  faculty  are  capable  of  giving.  Experts  argue  the  need  for  a 
stand-alone  technology  course  as  part  of  the  preservice  curriculum  to  adequately  prepare  new  teachers  for  new  roles 
versus  the  modeling  of  technology  integration  across  the  preservice  program.  While  a stand-alone  technology  course 
should  provide  preservice  teachers  with  adequate  skills  to  meet  technology  standards,  university  and  college 
administrations  are  reducing  program  credit  hour  requirements  to  assist  students  in  graduating  in  a timely  manner. 
Few  programs  can  still  afford  the  credit  hours  to  devote  to  a stand-alone  technology  course.  Whatever  the  solution, 
instructional  technology  departments  are  often  housed  outside  of  initial  certification  departments,  and  face 
challenges  to  influence  initial  certification  program  curriculum.  Relationships  of  an  increasing  cooperative  nature 
are  called  for. 

At  Georgia  State  University,  the  Instructional  Technology  unit  has  been  working  closely  with  the  Middle 
Childhood  Education  unit  for  the  past  three  years  to  develop  just  such  a cooperative  relationship.  Working  together, 
the  units  have  re-designed  the  stand-alone  technology  course  into  an  innovative,  alternative  approach  to  technology 
in  teacher  education  in  which  introductory  teaching  methods  are  taught  in  a technology-rich  learning  environment. 
In  addition,  a multi-submission  portfolio  assessment  plan  for  all  Middle  Childhood  Education  students  was 
instituted  to  ensure  that  all  students  meet  multiple  national  standards  prior  to  graduation. 

This  paper  explores  the  process  and  the  outcomes  of  this  partnership.  Current  and  future  plans  for  the 
partnership  are  provided.  In  addition,  the  authors  provide  their  personal  perceptions  why  this  partnership  worked, 
and  continues  to  grow.  Finally,  recommendations  for  establishing  partnerships  between  IT  units  and  initial 
certification  units  are  provided. 

Instructional  Technology  and  Teacher  Education 

Computer  technology  has  been  available  for  use  in  educational  settings  for  several  decades.  According  to  a 
survey  of  state  technology  officials  (Trotter,  1999),  42  states  require  teacher  preparation  programs  to  include 
technology.  One  might  think  that  by  this  time  colleges  of  education  (COEs)  are  successfully  preparing  teachers  to 
integrate  technology  into  instructional  practices.  However,  this  has  not  necessarily  been  the  case.  In  1995,  the  Office 
of  Technology  Assessment  (OTA)  published  a report  on  the  state  of  teachers  and  technology.  According  to  the 
OTA,  teachers  were  not  and  did  not  feel  adequately  prepared  to  integrate  technology  into  their  teaching  practices. 
One  of  the  contributing  factors  cited  was  the  lack  of  technology  training  available  in  teacher  preparation  programs  at 
colleges  of  education  (COE).  When  technology  instruction  was  provided,  it  involved  teaching  about  technology  not 
teaching  with  technology.  In  most  instances,  COE  faculty  did  not  model  technology  integration  with  their  preservice 
students.  Willis  and  Mehlinger  (1996)  conducted  a literature  review  on  technology  and  teacher  education.  Their 
findings  concurred  with  the  OTA  report: 
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Most  preservice  teachers  know  very  little  about  effective  use  of  technology  in  education  and 
leaders  believe  there  is  a pressing  need  to  increase  substantially  the  amount  and  quality  of 
instruction  teachers  receive  about  technology.  The  idea  may  be  expressed  aggressively, 
assertively,  or  in  more  subtle  forms,  but  the  virtually  universal  conclusion  is  that  teacher 
education,  particularly  preservice,  is  not  preparing  educators  to  work  in  a technology-enriched 
classroom  (p.  978). 

According  to  a recent  survey  of  416  teacher  preparation  institutions  commissioned  by  the  Milken  Exchange 
of  Education  Technology,  most  faculty  members  did  not  model  the  use  of  instructional  technology  skills  in  their 
teaching  (Moursund  & Bielefeldt,  1999).  In  several  studies  it  appears  that  faculty  who  are  not  modeling  are  also  not 
requiring  students  to  use  technology  in  their  lessons  or  assignments  (Lewallen,  1998;  U.S.  Congress,  1995;  Wetzel, 
1993). 

In  a nationwide  survey  of  education  majors  and  faculty,  Fulton  (1989)  found  that  while  58  percent  of  the 
faculty  thought  that  graduates  certified  in  secondary  education  were  well  prepared  to  use  technology,  only  29 
percent  of  the  students  felt  they  were.  However,  a report  produced  by  the  U.S.  Department  of  Education  (2000) 
revealed  refreshing  news:  less  experienced  teachers  were  more  likely  than  experienced  colleagues  to  indicate  that 
college  course  work  prepared  them  to  use  computers  in  their  classrooms.  “84  percent  of  teachers  with  3 or  fewer 
years  and  76  percent  of  teachers  with  4 to  9 years  of  teaching  experience  reported  that  college/graduate  work 
prepared  them  to  use  these  technologies  to  any  extent,  compared  with  44  percent  of  teachers  with  10  to  19  years  and 
31  percent  of  teachers  with  20  or  more  years  of  teaching  experience”  (p.  78).  While  teacher  education  programs  still 
face  obstacles  as  they  prepare  preservice  teachers,  it  is  evident  they  are  making  in-roads. 

Models  of  IT  Instruction  in  Teacher  Education  Programs 

These  in-roads  are  being  made  via  stand-alone  computer  courses  as  well  as  through  integrated  coverage 
across  teacher  education  curriculum.  Early  efforts  to  infuse  technology  into  teacher  education  often  resulted  in  a 
stand-alone  course  that  focused  primarily  on  technology  literacy  skills.  Many  of  the  strategies  used  were  based  on  a 
behavioral  model  in  which  students  focused  on  learning  a prescribed  set  of  skills  and  were  assessed  through 
objective  computerized  assessments.  It  was  not  uncommon  for  these  courses  to  be  taught  by  technology  faculty  with 
little  input  from  education  faculty  (Willis  & Mehlinger,  1996).  In  this  model,  teaching  and  technology  are  separated. 
Therefore  preservice  teachers  are  not  able  to  integrate  technology  into  their  teaching  practices. 

This  model  is  still  used  in  many  COEs.  Leh  (1999)  conducted  a study  on  the  content  of  technology  courses 
offered  to  education  majors  at  25  colleges  and  universities.  The  study  revealed  that  while  all  of  the  courses  focused 
on  concepts  and  skills,  only  52  percent  taught  about  curriculum  integration.  Results  of  study  by  Bennett  and  Daniel 
(1999)  on  novice  teachers  who  experienced  a stand-alone  course  indicated  that  having  only  a single  course  in 
computer  technology  was  not  sufficient.  It  did  not  adequately  prepare  teachers  to  apply  technology  in  the  classroom. 

A second  model  of  instructional  technology  instruction  in  teacher  education  programs  is  one  in  which  all 
teacher  education  faculty  model  technology  integration  across  all  courses.  While  ideal  in  concept,  it  is  arguable  that 
many  teacher  education  faculty  still  lack  sufficient  technology  skills  and  access  to  successfully  practice  cross- 
program modeling  of  technology  integration. 

In  response,  some  COEs  have  re-invented  the  stand-alone  course  to  make  it  more  constructivist  in  nature 
with  a greater  focus  on  technology  integration.  Effective  teacher  education  programs  combine  this  course  with 
technology  integration  in  the  teacher  education  courses.  In  a follow-up  study  to  the  OTA  report,  Wetzel  and  Strudler 
(1999)  looked  at  four  colleges  deemed  exemplary  in  their  approaches  to  prepare  inservice  teachers  to  use 
technology.  The  study  indicated  that  each  of  these  programs  had  a required  educational  technology  class  for 
preservice  teachers  to  take  early  in  their  program.  In  addition,  each  institution  was  part  of  a larger  plan  for  preparing 
students  to  teach  with  technology.  The  Milken  Exchange  on  Education  Technology  report  (1999)  calls  for  increased 
use  of  technology  in  curriculum  courses.  It  too  indicated  that  a single  course  in  instructional  technology  does  not 
provide  adequate  training  for  preservice  teachers.  This  third  model,  the  integrated  approach  along  with  a required 
technology  for  teachers  course,  may  be  the  best  approach,  particularly  in  light  of  the  renewed  focus  on 
accountability  in  teacher  education. 

Accountability  in  Teacher  Preparation 

There  is  a national  movement  towards  accountability  in  teacher  preparation  programs.  New  technology 
standards  for  teachers  along  with  revised  accreditation  requirements  will  require  teacher  preparation  programs  to 
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more  closely  examine  the  ability  of  their  new  teacher  candidates  to  teach  with  technology.  Recently  published 
National  Education  Technology  Standards  for  Teachers  (NETS-T)  (International  Society  for  Technology  in 
Education,  2000)  reflect  this  movement.  Along  with  the  standards,  the  International  Society  for  Technology  in 
Education  (ISTE)  has  created  “professional  preparation  performance  profiles.”  These  profiles  provide  scenarios  for 
the  types  of  activities  that  teacher  preparation  programs  can  expect  from  their  students  at  four  phases  of  professional 
development  from  general  preparation  through  their  first  year  teaching.  This  publication  is  timely  and  comes  on  the 
heel  of  a call-to-action  to  the  COEs  by  the  National  Council  for  Accreditation  of  Teacher  Education  (NCATE). 

In  1997,  NCATE  concluded  that  a majority  of  teacher  education  programs  were  not  doing  what  needed  to 
be  done  in  terms  of  preparing  teachers  to  teach  in  the  21st  century  classrooms.  NCATE  recommended  that  its 
accreditation  body  recognize  technology  education  for  teachers  as  central  to  the  teacher  preparation  process.  As  a 
result,  NCATE  raised  the  bar.  Aligned  with  Interstate  New  Teacher  Assessment  and  Support  Consortium  (INTASC) 
standards,  the  newest  NCATE  unit  standards  now  require  teacher  candidates  to  be  able  to  “appropriately  and 
effectively  integrate  technology  and  information  literacy  in  instruction  to  support  student  learning”  (2000,  p.  8). 

States  are  also  calling  for  accountability.  In  our  state  of  Georgia,  the  Board  of  Regents  of  the  University 
System  of  Georgia  has  adopted  Guiding  Principles  on  Teacher  Preparation  ( USG  News:  Principles  on  Teacher 
Preparation  Approved , April,  1998).  This  policy  “ ‘guarantees’  the  performance  of  P-12  teachers  prepared  through 
[the  University  System’s]  teacher  education  programs  for  teachers  who  are  teaching  within  the  fields  for  which  they 
have  been  prepared”  (p.l).  Under  the  guarantee  principle,  “the  University  System  will  ‘take  back’  any  teacher  within 
the  first  two  years  after  graduation  from  a System  institution  when  a school  district  in  Georgia  determines  the 
teacher’s  performance  is  less  that  effective  in  helping  students  make  satisfactory  progress.. .If  taken  back,  a teacher 
will  receive  additional  preparation  at  no  cost  to  the  teacher  or  to  the  school  district”  ( USG  News:  Principles  on 
Teacher  Preparation,  March,  1998,  p.  1). 

In  addition,  Georgia  Governor  Roy  Barnes  appointed  an  Education  Reform  Study  Commission  to  look  at 
ways  to  improve  Georgia’s  schools.  The  results  of  the  study  created  the  A Plus  Education  Reform  Act  of  2000  , 
passed  into  law  earlier  this  year  (2000).  Out  of  the  act  came  two  technology-related  initiatives  that  impact  teachers 
and  teacher  preparation  programs.  First,  the  act  mandates  that  renewable  teaching  certificates  would  not  be  granted 
unless  the  candidate  demonstrated  “...satisfactory  proficiency  on  a test  of  oral  and  written  communication  skills,  a 
test  of  computer  skill  competency,  [underline  added]  and  an  assessment  to  demonstrate  satisfactory  on-the-job 
performance  appropriate  to  the  applicant’s  field  of  certification”  (p.  65).  Second,  the  act  holds  teacher  preparation 
programs  at  universities  and  colleges  responsible  for  their  graduates’  technology  competencies.  Universities  and 
colleges 


shall  require  students  in  such  programs  to  be  proficient  in  computer  and  other  instructional 
technology  applications  and  skills  including  understanding  desktop  computers,  their  applications, 
integration  with  teaching  and  curriculum,  and  their  utilization  for  individualized  instruction  and 
classroom  management.  There  shall  be  a test  to  assess  the  proficiency  of  students  enrolled  in 
teacher  preparation  programs  in  computer  and  other  instructional  technology  applications  and 
skills,  (p.  68). 

An  Alternative  Approach:  Cooperative  Faculty  Partnerships 

In  considering  how  to  best  address  these  accountability  issues,  Georgia  State  University  explored 
alternative  approaches  to  technology  instruction  as  well  as  how  IT  faculty  might  be  involved  in  preservice  programs. 
The  development  of  this  alternative  approach  was  made  possible  by  a collaborative  partnership  established  between 
the  Instructional  Technology  unit  and  the  Middle  Childhood  Education  unit.  This  partnership  was  developed  in  an 
effort  to  redesign  the  initial  certification  programs  at  GSU  to  meet  changing  course  offering  calendars  as  well  as  the 
call  for  increased  accountability  in  teacher  education  by  professional  associations  and  accrediting  agencies.  Other 
universities  have  also  examined  the  potential  of  collaborative  partnerships  between  instructional  technology  and 
initial  certification  programs.  Dufifield’s  (1997)  account  of  an  instructional  technology-teacher  education 
partnership  at  University  of  Colorado-Denver  chronicles  a four-year  journey  in  which  Duffield  served  as  an  IT 
consultant  to  the  elementary  methods  team.  What  is  telling  is  that  more  partnerships  haven’t  been  cited.  Perhaps  the 
answer  to  this  can  be  found  in  examining  how  colleges  of  education  are  typically  structured.  Instructional 
technology  programs  teach  to  a more  diverse  audience  than  do  initial  certification  programs,  and  as  such,  often  have 
difficulty  fitting  in  to  the  typical  college  of  education  structure.  Historically,  IT  programs  have  developed  from  two 
theoretical  foundations,  audio-visual/media,  and  corporate  training,  design,  and  development;  programs  which  focus 
on  a broader  than  K-12  audience.  Because  of  this  diverse,  non  K-12  heritage,  many  universities  have  difficulty 
placing  IT  programs  within  their  departmental  structure.  The  simple  solution  is  to  set  the  IT  unit  as  its  own 
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department.  While  this  solution  does  allow  the  IT  unit  a good  deal  of  autonomy,  it  does  have  its  drawbacks, 
particularly  when  everyone  else  in  the  college  has  a K-12  focus.  Barriers  can  go  up  quickly,  and  what  ensues  is  a 
lack  of  coordination  and  cooperation  between  IT  and  initial  preparation  programs.  The  other  popular  solution, 
housing  IT  with  other  broader  than  K-12  programs  such  as  curriculum  and  instruction,  educational  psychology,  or 
educational  leadership  departments,  has  also  not  been  conducive  to  fostering  partnerships  with  programs  that  offer 
initial  K-12  teacher  certification.  It  is  possible  that  this  division,  however  convenient  it  might  be  for  the  IT  training 
persona,  might  be  partially  responsible  for  the  lag  in  technology  integration  in  the  schools  and  in  our  preservice 
programs.  At  Georgia  State  University,  initial  certification  programs  fall  under  the  jurisdiction  of  the 

Professional  Education  Faculty,  a combination  of  faculty  of  the  College  of  Education  and  the  College  of  Arts  and 
Sciences.  The  IT  unit  was  moved  several  years  ago  to  the  comfortable  umbrella  of  the  largest  department  in  the 
College  of  Education,  Middle/Secondary  Education  and  Instructional  Technology  (MSIT).  The  MSIT  department 
prepares  teachers  in  a variety  of  traditional  and  alternative  programs  for  certification  in  Middle  Childhood  Education 
(grades  4-8)  and  Secondary  Education  (grades  9-12).  Although  the  IT  unit  was  housed  within  an  initial  preparation 
department,  for  several  years,  the  IT  unit  continued  to  address  the  broad  IT  audience,  and  until  1997,  served  an 
approximate  80%  corporate  audience.  It  was  at  this  same  time  that  several  factors  were  developing  to  force  a change 
not  only  in  the  focus  of  the  IT  unit,  but  also  of  the  MSIT  department. 

Program  Performance  Analysis  and  Formative  Evaluation 

In  the  mid-  and  late- 1990s,  several  national  organizations  introduced  and  promoted  standards  for  preservice 
teachers  and  their  programs  (Interstate  New  Teacher  Assessment  & Support  Consortium  principles,  International 
Society  for  Technology  in  Education  Technology  Standards  for  All  Teachers,  as  well  as  content  specific  standards). 
At  the  same  time,  the  University  System  of  Georgia  Board  of  Regents  determined  that  all  institutions  would  move 
from  a quarter  to  semester  calendar  beginning  with  the  1998-99  academic  year.  The  USG  Board  of  Regents 
guarantee  principle  mentioned  earlier,  as  well  as  pending  NCATE  and  APACE  (university-wide  Academic 
Programs  and  Continuing  Education  self-study)  reviews,  precipitated  the  entire  MSIT  department  to  participate  in  a 
program  performance  analysis  and  formative  evaluation.  In  examining  all  programs,  a culture  of  cooperation 
between  the  IT  unit  and  the  teacher  preparation  programs  was  established. 

With  an  opportunity  to  revamp  the  entire  Middle  Childhood  Education  program,  a Middle  Childhood 
Committee  (MCC)  was  formed.  This  committee  was  composed  of  faculty  representing  all  areas  of  study  for  the 
Middle  Childhood  Education  program:  language  and  literacy,  mathematics,  science,  social  studies,  reading,  and 
instructional  technology.  The  MCC  examined  all  required  guidelines  for  initial  preparation  programs  at  the  state  and 
national  levels.  Input  from  faculty  and  student  evaluations  and  surveys  were  also  incorporated  into  the  analysis.  All 
components  of  the  middle  childhood  undergraduate  program  were  analyzed:  program  admissions  and  exit  criteria, 
course  offerings,  course  experiences,  field  experiences,  scheduling  of  classes,  scheduling  of  student-cohort  groups 
and  faculty  teams.  As  a result,  major  program  changes  were  implemented.  This  paper  focuses  on  two  outcomes  of 
this  cooperative  relationship  which  effected  how  the  IT  unit  prepared  and  advised  preservice  teachers  and  interacted 
with  the  initial  preparation  programs:  the  redesign  of  the  stand-alone  technology  course  to  a technology-methods 
course  and  the  establishment  of  a standards-based  alternative  assessment  process  for  all  prospective  middle  grades 
teachers. 

Technology-Methods  Course  Development 

As  indicated  earlier,  many  teacher  education  programs  focused  on  either  a stand-alone  course,  or  on  a 
model  of  technology  infused  throughout  all  teacher  preparation  courses.  Some  schools,  including  GSU,  have  opted 
to  do  both.  Kovalchick  (1997)  offers,  “An  approach  that  I have  found  useful  is  to  blend  elements  from  both  a 
competency  based  models  and  integrative  models  into  a reflexive  approach  in  which  students  use  technology  as  both 
learner  and  teacher.  In  this  way,  preservice  teacher  education  students  are  challenged  through  direct  experience  to 
generate  personally  relevant  conceptions  of  technology”  (p.  31).  Smaldino  and  Muffoletto  (1997)  also  promote  a 
combination  approach.  “Our  model  attempts  to  blend  the  contents  of  the  existing  single  course  with  the  need  to 
nurture  technology  applications  within  methods  and  other  courses.  Thus,  students  first  gain  an  understanding  of  the 
applications  of  technology  in  education  in  the  broad  sense,  with  an  in-depth  examination  of  how  technology 
supports  learning  in  specific  content  areas”  (p.37). 

Prior  to  1997,  the  technology  course  at  GSU  was  a stand-alone,  skills-based  course  that  focused  on  the  use 
of  technology  as  a teacher  tool.  Content  included  such  technology  usage  as  word  processing,  mail  merging  a letter 
home  to  parents,  and  using  a spreadsheet  program  to  calculate  grades.  Little  to  no  learning  theory  or  instructional 
methods  were  included  in  the  lab-based  course.  In  addition,  the  technologies  covered  were  basic  in  nature  ~ 
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telecommunications  coverage  consisted  of  e-mail,  and  in  later  years,  the  Internet  as  a database  of  lesson  plans.  As 
pedagogy  played  virtually  no  role  in  the  course,  students  were  allowed  to  substitute  a passing  grade  on  a pencil  and 
paper  competency  test. 

In  1997,  at  the  request  of  the  Middle  Childhood  Committee,  the  standard  skills-based  preservice  technology 
course  underwent  a major  redesign.  In  the  first  year,  the  course  refocused  from  teacher-resource-based,  skills-based 
to  a technology-integration-into-the-curriculum  approach.  This  refocus  was  done  in  part  to  address  a potential  cause 
of  low  technology  adoption  in  preservice  teachers:  deficiencies  in  technology-integration  methods  (Leggett  & 
Persichitte,  1998). 

In  fall  semester  1998,  the  IT  unit  worked  with  the  MCC  to  redesign  the  course  to  further  situate  the  course 
content  in  teaching  methods.  While  maintaining  a lecture/lab  approach,  a WWW-based,  resource-based  learning 
environment  (RBLE)  was  introduced  as  part  of  the  course  (Hill,  1999;  Shoffner,  1999).  The  course,  and  its  related 
resource  laden  WWW  site,  incorporates  a problem-centered,  activity-based  approach  where  the  computer 
applications  are  anchored  in  authentic  and  familiar  contexts  in  which  teaching  and  learning  occurs  (Cognition  and 
Technology  Group  at  Vanderbilt,  1991;  Vygotsky,  1978).  This  approach  is  based  on  the  view  of  an  open  learning 
environment  in  which  learners  have  direct  input  on  the  direction  of  the  course  based  on  their  needs  (Hannafin,  1999; 
Hannafin,  Hall,  Land,  & Hill,  1994).  In  navigating  through  the  environment  and  tackling  challenges,  it  is  proposed 
that  students  will  also  develop  self-directed  learning  skills,  which  will  serve  them  well  as  they  enter  the  teaching 
profession.  Along  with  confidence  in  using  the  technology,  self-directed  learning  skills  have  been  identified  as  a 
characteristic  of  successful  technology-using  teachers  (Shoffner,  1996).  The  RBLE  can  be  accessed  at 
http://msit.gsu.edu/IT/32 1 Q/index.html 

At  the  same  time,  the  course  serves  as  an  introductory  teaching  methods  course,  introducing  preservice 
students  to  such  concepts  as  instructional  objectives,  lesson  planning,  evaluation,  and  assessment.  The  course  offers 
more  than  teaching  the  basic  ADDIE  instructional  design  model  as  a way  to  develop  lesson  plans  while  teaching 
about  technology  integration  skills.  In  the  Technology  for  Teachers  course  at  GSU,  the  technology  is  immersed  in 
learning  about  what  being  a teacher  entails  - briefly,  planning,  learning  theory,  instructional  strategies,  classroom 
management,  and  assessment.  Our  hope  is  that  by  introducing  the  technology  and  the  methods  together,  early  in  the 
program,  that  a)  students  will  forever  forward  view  technology  as  natural  to  the  learning  process  as  the  textbook  and 
the  pencil;  and  b)  both  the  technology  and  the  methods  will  be  reinforced  throughout  their  other  courses  at  GSU, 
One  way  in  which  continuity  and  reinforcement  occurs  is  in  the  use  of  portfolios  for  assessment.  In  the  Technology 
for  Teachers  course,  preservice  students  generate  a portfolio  documenting  the  design  of  technology-supported 
instructional  environment  that  facilitates  student  learning  through  the  design  and  development  of  student-centered 
learning  activities.  The  use  of  portfolio  development  and  assessment  continues  throughout  the  remainder  of  Middle 
Childhood  Education  program  of  study. 

MCE  Standards  Based  Alternative  Assessment 

A second  outcome  of  the  MCC  was  the  establishment  of  a continuous  process  of  portfolio  development  and 
assessment  for  all  students.  In  response  to  the  Board  of  Regents  guarantee  principle,  increasing  accountability  in 
teacher  preparation  programs,  and  the  Middle  Childhood  Committee’s  recommendation  to  strengthen  the  preservice 
teachers’  overall  professional  development,  the  committee  recommended  that  the  program  include  an  exit 
assessment  that  examined  the  student’s  ability  to  apply  what  they  learned  in  all  their  courses  in  some  cohesive 
manner.  After  examining  several  assessment  models,  both  traditional  and  alternative,  a portfolio  development 
process  with  benchmarks  throughout  the  program  and  final  submission  as  an  exit  requirement  was  adopted. 

Although  most  skills  and  concepts  are  developed  in  individual  courses,  it  is  important  that  preservice 
teachers  have  command  of  these  concepts  and  skills  with  knowledge  of  how  to  integrate  these  concepts  and  skills 
into  all  aspects  of  teaching.  Therefore,  a major  goal  of  portfolio  requirement  was  to  develop  the  preservice  students’ 
ability  to  integrate  several  components  of  the  program  across  all  courses  and  to  develop  knowledge  and  skills  in 
applying  these  components  in  all  aspects  of  teaching.  Among  key  skills  and  concepts  under  discussion  were: 
integrating  technology,  developing  and  implementing  lesson  plans  and  assessment  strategies,  developing  and 
implementing  a classroom  management  plan,  working  with  diverse  learners,  developing  as  reflective  practitioners, 
and  so  on.  After  a review  of  the  principles  of  the  Interstate  New  Teacher  Assessment  and  Support  Consortium 
(INTASC),  the  committee  agreed  that  the  principles  of  INTASC  encompassed  and  addressed  all  major  components 
of  the  middle  childhood  program  and  could  be  used  to  facilitate  the  development  of  the  preservice  teachers.  Thus  the 
committee  established  portfolio  guidelines  that  focused  on  the  ten  principles  of  INTASC.  The  INTASC  Principles 
are  included  below. 
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Principle  1 The  teacher  understands  the  central  concepts,  tools  of  inquiry,  and  structure  of  the  discipline(s) 
he  or  she  teaches  and  can  create  learning  experiences  that  make  these  aspects  of  subject  matter 
meaningful  for  students. 

Principle  2 The  teacher  understands  show  children  learn  and  develop  and  can  provide  learning 
opportunities  that  support  their  intellectual,  social,  and  personal  development. 

Principle  3 The  teacher  understands  how  students  differ  in  their  approaches  to  learning  and  creates 
instructional  opportunities  that  are  adapted  to  diverse  learners. 

Principle  4 The  teacher  understands  and  uses  a variety  of  instructional  strategies  to  encourage  students’ 
development  of  critical  thinking,  problem  solving,  and  performance  skills. 

Principle  5 The  teacher  uses  an  understanding  of  individual  and  group  motivation  and  behavior  to  create  a 
learning  environment  that  encourages  positive  social  interaction,  active  engagement  in 
learning,  and  self-motivation. 

Principle  6 The  teacher  uses  knowledge  of  effective  verbal,  nonverbal,  and  media  communication 

techniques  to  foster  active  inquiry,  collaboration,  and  supportive  interaction  in  the  classroom. 

Principle  7 The  teacher  plans  instruction  based  on  knowledge  of  subject  matter,  students,  the  community, 
and  curriculum  goals. 

Principle  8 The  teacher  understands  and  uses  formal  and  informal  assessment  strategies  to  ensure  the 
continuous  intellectual,  social,  and  physical  development  of  the  learner. 

Principle  9 The  teacher  is  a reflective  practitioner  who  continually  evaluates  the  effects  of  his  or  her 
choices  and  actions  on  others  (students,  parents,  and  other  professionals  in  the  learning 
community),  and  who  actively  seeks  out  opportunities  to  grow  professionally. 

Principle  10  The  teacher  fosters  relationships  with  school  colleagues,  parents,  and  agencies  in  the  larger 

community  to  support  students’  learning  and  well-being. 


Table  l:  Interstate  New  Teacher  Assessment  and  Support  Consortium  Principles  (Council  of  Chief  State  School 
Officers,  1999) 

Through  the  continuous  collaboration  of  the  middle  childhood  committee,  guidelines  for  portfolio 
development  were  documented,  benchmarks  were  established,  implementation  procedures  were  outlined,  and  an 
assessment  instrument  and  procedures  were  designed.  The  committee  reviewed  course  syllabi  for  all  MCE 
undergraduate  education  courses  to  determine  which  INTASC  principles  were  met  in  each  course.  The  principles 
were  aligned  with  the  program’s  schedule  of  course  sequence  and  experiences  to  establish  which  principles  the 
preservice  students  would  be  able  to  address  at  established  intervals.  These  intervals  serve  as  benchmarks  to  assess 
the  students’  portfolios. 

The  committee  decided  that  the  preservice  students  would  write  a narrative  for  each  principle  and  provide 
artifacts  to  substantiate  their  knowledge,  growth  and  experiences  in  the  program.  In  the  narrative,  students  are 
required  to  discuss  personal  accounts  that  address  all  concepts  within  the  principle.  In  that  the  student  is  required  to 
address  all  concepts  of  the  principle  in  the  narrative,  it  was  clear  that  a specific  artifact  might  address  only  one  or 
two  concepts  within  a principle.  Therefore  the  students  are  required  to  explain  within  the  narrative  how  the  artifact 
addresses  a specific  concept. 

The  committee  established  benchmarks  based  on  experiences  acquired  within  the  prescribed  course 
sequence.  Students  are  expected  to  complete  all  content  courses  prior  to  the  senior  year  in  the  program.  The 
University  System  of  Georgia  Board  of  Regents  requires,  within  a 120-hour  semester  program,  that  middle 
childhood  education  (MCE)  majors  have  two  content  areas  of  concentration  --  12  semester  hours  in  a major  area  and 
nine  semester  hours  as  a minor  area.  During  the  junior  and  senior  years  the  preservice  teachers  are  immersed  in 
teacher  education  courses  that  include  field  experience  components.  The  INTASC  principles  and  benchmarks  were 
aligned  with  the  Professional  Studies  and  Student  Teaching  coursework:  introduction  to  middle  schools, 
instructional  technology,  teaching  reading  block,  topics  courses  in  the  content  areas,  methods  block,  diversity  course 
and  student  teaching.  The  committee  established  the  following  schedule  as  benchmarks  for  assessing  student  growth 
and  development  in  the  program.  Upon  completion  of  the  Professional  Studies  courses  (at  the  end  of  the  junior 
year),  the  MCE  students  are  assessed  for  meeting  INTASC  Principles,  1,2, 6, 7, 8,  and  9.  Prior  to  entrance  to  student 
teaching  (midway  in  the  senior  year),  MCE  students  submit  portfolios  demonstrating  competency  for  all  ten 
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principles.  Next  the  committee  established  a system  for  portfolio  evaluation,  introducing  the  students  to  the  process 
through  seminars  and  coursework,  and  assigning  faculty  advisors  to  assist  students. 

Students  are  guided  through  the  portfolio  process.  Early  in  the  semester  in  which  students  begin 
Professional  Studies  course,  seminars  are  delivered  by  the  MCC  to  introduce  the  portfolio  process  to  the  preservice 
students.  The  MSIT  Middle  Childhood  Education  Program  Portfolio  Evaluation  Guide  (Many  et  al , 1998) 
introduces  students  to  the  INTASC  principles,  and  explains  the  portfolio  assembly  and  evaluation  process.  The 
Converting  Your  IT  3210  Learning  Environment  Portfolio  To  The  Junior-Year  MCE  Program  Portfolio  Guide 
(Shoffher  et  al , 1998)  presents  strategies  for  reformatting  the  Learning  Environment  Portfolio  produced  in  the 
Technology  for  Teachers  course  to  the  Professional  Studies  Portfolio. 

The  portfolio  is  accepted  in  a variety  of  formats.  Students  may  submit  an  electronic  portfolio  (on  compact 
disc),  a website,  or  a notebook  for  faculty  review.  (The  majority  of  students  in  program  continue  to  favor  the 
notebook  version.)  Upon  portfolio  submission,  the  MCC  meets  and  collaboratively  assesses  each  portfolio.  A simple 
rubric  is  used  to  assess  competency  in  regard  to  INTASC  Principles.  Faculty  reviewers  indicate  whether  each 
principle  was  “not  met,”  “met,”  or  “met  in  an  exceptional  manner,”  and  give  feedback  on  the  documentation  of  each 
principle.  Students  receiving  a score  of  “not  met”  on  any  principle  are  required  to  meet  with  a faculty  advisor  to 
discuss  what  must  be  accomplished  to  achieve  successful  experiences  and  documentation  for  the  principle. 

The  portfolio  review  process  was  implemented  in  the  fall  1998  semester.  Due  to  the  iterative  nature  of  the 
assessment  process,  all  students  met  all  principles  prior  to  graduation.  The  portfolios  generated  by  students  at  the 
close  of  their  coursework  consistently  demonstrated  a clear  understanding  of  the  theoretical  underpinnings  and 
application  of  teaching  and  learning  knowledge.  Student  narratives  provided  rich  and  reflective  insight  into  how 
each  preservice  teacher  was  able  to  apply  what  was  learned  in  the  college  classroom  to  the  middle  grades  classroom. 
While  students  were  initially  resistant  to  the  added  work  of  compiling  the  portfolio,  by  the  end  of  their  program, 
students  enthusiastically  espoused  the  benefits  of  the  portfolio  process  in  allowing  them  to  compose  a holistic  vision 
of  their  preparation  and  educational  philosophy,  as  well  as  the  ability  to  articulate  this  vision.  Many  students 
comment  on  the  benefits  of  the  portfolio  process  in  preparing  them  to  successfully  interview  for  permanent 
employment. 

Students  in  the  first  cohort  to  complete  the  portfolio  process  are  now  certified  educators  employed  in  the 
schools.  Several  research-based  initiatives  are  underway  to  examine  their  preparedness  as  inservice  teachers.  In 
addition,  a study  is  in  progress  that  will  examine  the  “INTASC”  portfolios  for  the  demonstration  of  technology 
competencies  (NETS-T  Profiles).  The  MCC  committee  continues  to  formatively  evaluate  their  program  in  light  of 
national  and  state  directives,  as  well  as  student  needs. 

Collaboration  & Cooperation:  Contributing  Factors  at  GSU 

The  authors  would  be  remiss  if  we  did  not  reflect  on  our  case  study  to  determine  what  factors  may  have 
contributed  to  our  success,  and  from  that  reflection  make  suggestions  on  how  IT  units  at  other  COEs  might  do  the 
same.  Our  reflection  produced  three  core  factors  that  contributed  to  the  success  of  our  collaboration:  the  nature  of 
middle  grades,  a committed  faculty,  and  a culture  of  mutual  respect  within  the  committee,  the  department,  the 
college,  and  the  professional  education  faculty. 

It  is  the  nature  of  those  who  teach  at  the  middle  school  level  to  be  cognizant  of  multiple  disciplines  as  well 
as  flexible.  Middle  schools  typically  employ  a teaming  approach  to  instruction,  where  students  are  assigned  to  a 
team  of  teachers  who  cover  the  core  subjects.  To  operate  successfully  in  the  team  structure,  middle  school  teachers 
must  be  flexible  and  cooperative.  This  flexibility  and  cooperativeness  must  also  be  present  in  those  who  prepare 
middle  school  teachers.  Furthermore,  teacher  licensure  at  the  middle  grades  level  is  across  all  content  areas. 
Although  preservice  teachers  prepare  in  a major  and  a minor  content  field,  they  are  licensed  to  teach  all  fields,  and 
must  be  ready  to  teach  in  any  of  the  four  core  content  areas  and  reading.  Although  it  is  possible  to  receive  an 
advanced  graduate  degree  in  Middle  Childhood  Education,  most  faculty  members  teaching  in  our  program  are  from 
a specialty  content  area  (mathematics,  language  and  literacy,  reading,  science,  social  studies,  or  instructional 
technology).  It  is  imperative  that  those  who  prepare  middle  grades  teachers  work  together  to  facilitate  this  broad 
multidisciplinary  preparation. 

A second  factor  that  contributed  to  the  success  of  the  partnership  at  GSU  is  the  nature  of  the  faculty. 
Although  the  faculty  differed  in  their  fields  of  specialty  and  their  experience  in  the  K-12  and  college  level,  all  of  the 
faculty  involved  in  the  Middle  Childhood  Committee  were  committed  to  making  this  program  work.  Georgia  State 
University  has  a long  history  of  preparing  outstanding  middle  school  educators,  and  the  faculty  was  and  is 
committed  to  continuing  this  tradition.  The  committee  met  regularly,  at  times  weekly,  to  plan  the  program,  the 
technology  methods  course,  and  the  portfolio  assessment  process.  One  reason  for  the  MCC’s  commitment  level  was 
that  they  were  given  ownership  of  the  program  by  a very  supportive  department  administration.  Committee 
members  continue  to  give  their  time  to  meet  and  review  portfolio  submissions  each  semester. 
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The  third  factor  contributing  to  the  partnership’s  success  was  the  establishment  of  a culture  of  mutual 
respect  among  the  committee  members.  As  committee  members  come  from  a variety  of  disciplines,  each  had 
something  to  bring  to  the  table.  Early  on,  the  IT  faculty  members  on  the  committee  were  able  to  establish  their 
credibility  as  educators.  All  content  areas,  including  instructional  technology,  were  considered  equally  important  to 
the  preparation  of  new  teachers. 

Suggestions  For  Establishing  Partnerships 

Although  the  IT  unit  at  GSU  is  strategically  placed  to  facilitate  such  collaborative  partnerships,  the  authors 
believe  that  some  steps  can  be  taken  to  nurture  such  partnerships,  even  when  the  IT  unit  is  housed  outside  the  initial 
preparation  programs.  We  offer  the  following  suggestions  to  establish  cooperative  partnerships  with  teacher 
education  program  units. 

First,  instructional  technology  faculty  members  who  wish  to  work  with  teacher  education  programs  must 
become  familiar  with  current  issues  in  teacher  education  preparation  and  in  K-12  schools.  As  it  is  possible  or  even 
likely  that  an  IT  faculty  member  may  not  be  a certified  K-12  teacher,  other  steps  may  be  taken  to  develop  an 
understanding  of  schools.  IT  faculty  members  can  volunteer  to  collaborate  with  a K-12  teacher,  designing  and  team 
teaching  a unit  of  study.  Serving  on  school  technology  committees  is  yet  another  way  IT  faculty  can  develop  and 
understanding  of  the  K-12  school  culture. 

Second,  the  IT  unit  should  ideally  find  a single  teacher  preparation  unit  or  team  that  is  willing  to  work  with 
an  IT  consultant.  Many  in  IT  would  argue  that  the  integration  of  technology  should  take  place  in  a systemic  fashion. 
However,  an  incremental  approach  is  more  likely  to  be  successful,  and  in  this  instance,  success  will  likely  breed 
more  success.  In  short,  pick  a single  program  with  whom  to  establish  a rapport,  and  then  work  on  establishing  a 
relationship. 

To  nurture  this  budding  relationship,  it  is  essential  that  the  IT  faculty  member(s)  attend  teacher  education 
department  or  unit  meetings.  It  is  at  these  formal  meetings  that  the  IT  faculty  can  establish  their  credibility  as 
educators  by  providing  information  on  technology  integration  strategies  while  also  gamering  information  about  the 
certifying  program.  Duffield  (1997)  concurs,  “Probably  the  most  important  element  of  the  second  year  was  the  time 
I spent  planning  and  working  with  the  elementary  methods  team.  I was  able  to  become  familiar  with  the  content  and 
methods  they  used  and  begin  discussions  about  how  technology  could  be  integrated  into  the  courses.  I also  served  as 
an  advocate  for  technology,  keeping  it  part  of  every  discussion”  (p.  24).  In  order  to  serve  as  an  advocate  for 
technology,  IT  faculty  must  stay  current  with  research  and  methods  in  instructional  technology  integration 
strategies. 

Conclusions 

Accountability  directives  for  new  teacher  preparedness  are  not  likely  to  go  away  any  time  soon. 
Instructional  technology  preparation  will  likely  continue  to  be  a critical  issue  in  teacher  education  for  many  years  to 
come.  Instructional  technology  units  can  no  longer  teach  only  to  their  corporate  training  design  and  development 
roots.  For  colleges  of  education  to  successfully  prepare  teachers  for  the  21st  century,  instructional  technology  will 
need  to  be  more  cohesively  included  in  teacher  preparation  programs.  It  is  imperative  that  more  cooperative 
partnerships  be  established  between  instructional  technology  units  and  initial  preparation  programs.  The  authors 
encourage  IT  units  to  initiate  and  nurture  these  partnerships,  making  possible  more  innovative  approaches  to  this 
important  field  of  study. 
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Toward  a Post-Modern  Agenda  in  Instructional  Technology 


David  L.  Solomon 
Wayne  State  University 


Abstract 

“Post-modernists  look  to  the  past  and  future  equally  and  position  themselves  in  the  present,  seeing  time  as 
a broken  continuum  in  need  of  acknowledgement " (Jencks,  1992b,  p.  6). 

Traditional  views  in  instructional  technology  are  often  based  upon  the  application  of  scientific  knowledge. 
Post-modernism,  an  alternative  paradigm,  questions  whether  science  alone  offers  the  best  approach  to  teaching  and 
learning.  Post-modernism  holds  promise  for  guiding  research  and  development  in  instructional  technology ; 
however,  its  potential  contributions  to  the  field  require  clarification.  Accordingly,  cogent  definitions  of  post- 
modernism have  been  constructed  --  from  an  instructional  technologist’s  point  of  view  - and  implications  for  the 
field  have  been  presented  to  address  the  concerns  of  critics.  This  paper  received  the  2000  ETR&D  Young  Scholar 
Award  and  the  author  notes  that  the  term,  “post-modern,  ” shall  be  hyphenated  to  symbolize  "...  the  continuation  of 
Modernism  and  its  transcendence  ’’  (Jencks,  1995,  p.30). 

Throughout  its  history,  the  field  of  instructional  technology  has  been  focused  on  improving  teaching  and 
learning  through  technology  (Ely,  1999;  Saettler,  1990).  Over  the  years,  various  interpretations  of  the  word, 
“technology,”  have  been  at  the  root  of  inquiry  and  debate  in  our  field.  As  a result,  traditional  and  alternative 
approaches  to  theory  development  and  practice  have  emerged.  A traditional  interpretation  of  technology  is  the 
systematic  application  of  science  (Clark  & Estes,  1998)  which  emphasizes  the  utilization  of  scientific  knowledge 
and  principles  (Seels  & Richey,  1994).  Grounded  in  the  behavioral  sciences,  traditional  approaches  to  instructional 
technology  share  common  values  including  the  use  of  precision-based  methods,  measurement,  replicability, 
predictability  and  order  (Gustafson  & Tillman,  1991;  Jonassen  et  al.,  1997;  Seels  & Richey,  1994).  Alternative 
perspectives  in  our  field  assume  a broader  interpretation  of  technology  as  the  systematic  application  of  all  sources  of 
organized  knowledge  (e.g.,  literature,  science,  the  arts),  suggesting  that  art,  craft  and  science  all  have  roles  to  play  in 
instructional  technology  (Davies,  1981/1991;  Richey,  1995;  Seels,  1995).  Whereas  traditional  views  in  our  field 
rely  solely  upon  scientific  knowledge,  post-modernism,  an  alternative  paradigm,  questions  whether  or  not  science 
alone  offers  the  best  path  (Wilson,  1997a).  Within  the  past  10  years,  post-modern  perspectives  in  instructional 
technology  have  been  receiving  greater  attention  within  the  Association  for  Educational  Communication  and 
Technology  (Wilson,  1997a).  In  addition,  a growing  number  of  publications  in  the  field  have  begun  to  define  a 
post-modern  agenda.  While  the  Handbook  of  Research  for  Educational  Communications  and  Technology  (Jonassen, 

1996)  included  postmodern  theory  as  a foundation  for  research  in  our  field,  the  post-modern  agenda  remains 
unclear.  Agenda-building  is  a process  through  which  problems  or  issues  receive  professional  attention  (Richey, 

1997) .  Within  this  agenda-building  framework,  ideas  that  influence  professional  communities  are  aligned  with  the 
broader  agendas  of  the  field  (Richey,  1997).  Therefore,  communication  is  an  important  part  of  the  support  building 
process,  particularly  in  theory  development  (Richey,  1997).  Post-modernism  offers  alternative  perspectives  on  the 
theory  and  practice  of  instructional  technology;  however,  agenda  status  in  our  field  will  never  be  achieved  until 
communication  can  be  facilitated.  There  are  two  major  impediments  to  building  a post-modern  agenda  in 
instructional  technology:  First,  a widely  accepted  definition  of  post-modernism  does  not  exist.  Second,  the 
implications  for  theory  construction  are  unclear  (perhaps  a consequence  of  poor  definition).  Therefore,  the  purpose 
of  this  paper  is  to  construct  an  integrative  and  cogent  definition  of  post-modernism  that  will  initiate  dialogue  among 
professionals  in  our  field.  There  are  several  inherent  challenges  to  accomplishing  this  goal.  Since  post-modern 
discourse  is  mostly  abstract  and  obscure,  it  becomes  necessary  to  simplify  subject  matter  and  render  it  less  complex 
for  our  professional  community.  This  reductionistic  process  is  fundamentally  opposed  to  post-modern  principles.  In 
addition,  this  paper  summarizes  the  literature  that  has  been  written  about  post-modernism  across  a variety  of 
disciplines;  and,  while  it  is  always  important  to  see  the  tip  of  an  iceberg  while  navigating  through  uncharted 
territories,  we  must  also  acknowledge  the  presence  of  something  that  has  far  greater  depth. 

Part  One  of  this  paper  will  address  several  issues  impacting  the  definition  of  post-modernism.  Part  Two 
will  construct  a cogent  definition  of  post-modernism  that  will  promote  communication  within  the  field;  and,  Part 
Three  will  explore  the  potential  contributions  of  post-modern  concepts  in  instructional  technology.  Despite  the 
challenges,  this  paper  was  written  with  the  hopes  of  taking  a step  forward  in  the  support-building  process. 
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Part  One:  Issues  Impacting  the  Definition 

One  major  source  of  confusion  regarding  post-modernism  has  been  the  elusive  language  of  post-modern 
discourse.  Walter  Truett  Anderson  (1995b)  offers  the  following  insight  into  this  dilemma: 

The  postmodern  era  has  given  the  world  some  really  good  ideas  and  some  really  bad  writing. 

From  Derrida  on  down  to  humble  troopers  in  the  trenches  of  academia,  a style  that  has  come  to 
prevail  among  postmodernists  is  one  of  endless  complexification  and  obscurity.  The  general  idea 
seems  to  be  that  the  surest  way  to  establish  yourself  as  a profound  thinker  is  to  make  it  very 
difficult  for  anybody  to  understand  what  your  are  saying  (p.  92). 

Many  critics  are  disturbed  by  the  conventional  language  of  post-modern  discourse,  including  Hooks  (1995),  who 
asserts  that  “as  a discursive  practice  it  is  dominated  primarily  by  the  voices  of  white  male  intellectuals  and/or 
academic  elites  who  speak  to  and  about  one  another  with  coded  familiarity”  (p.  1 18). 

In  addition,  the  term  “post-modern”  is  represented  in  various  ways  in  the  literature.  The  phrase  is  spelled 
post-Modem  when  it  refers  to  a historical  epoch  (Jencks,  1992a).  Literary  criticism  often  styles  it  as  postmodern; 
and,  “ ...in  architecture  and  as  a cultural  event  it  is  usually  Post-Modem  (to  indicate  a doubly-coded  movement)” 
(Jencks,  1992a,  p.  16).  Jencks  (1995)  hyphenates  the  term  to  reflect  “...that  paradoxical  dualism,  or  double  coding, 
which  its  hybrid  name  entails:  the  continuation  of  Modernism  and  its  transcendence”  (p.  30).  Toulmin  (1985)  offers 
additional  insight  into  the  term: 

We  must  reconcile  ourselves  to  a paradoxical-sounding  thought:  namely,  the  thought  that  we  no 
longer  live  in  the  " modern  " world . The  “modem”  world  is  now  a thing  of  the  past.  Our  own 
natural  science  today  is  no  longer  “modem”  science.  Instead  ...  it  is  rapidly  engaged  in  becoming 
“post-modem”  science:  the  science  of  the  “postmodern”  world,  of  “postnationalist”  politics  and 
“postindustrial”  society  - the  world  that  has  not  yet  discovered  how  to  define  itself  in  terms  of 
what  it  is,  but  only  in  terms  of  what  it  has  just-now  ceased  to  be  (p.  254). 

Toulmin’s  logic  may  explain  some  of  the  ambiguity  associated  with  post-modernism  because  the  term  specifies  a 
departure  point,  but  leaves  the  final  destination  open-ended  (Jencks,  1992a). 

Post-Modernism  is  Evolving 

Any  attempt  to  define  post-modernism  is  a daunting  task  for  several  reasons.  First,  the  post-modem  agenda 
continues  to  evolve  and  is  ever-changing  (Jencks,  1992a).  Next,  post-modem  thought  includes  contributions  from 
many  areas  including  philosophy,  literature,  the  sciences  and  the  arts  (Doll,  1993;  Jencks,  1992b);  and,  perhaps  as  a 
result,  the  word  “post-modernism”  means  different  things  to  different  people  (Anderson,  1995b).  Jencks  (1992a) 
suggests  that  there  are  various  post-modem  movements  (e.g.,  feminism,  the  green  and  ecological  movement, 
libertarian  theology)  which  contribute  to  the  overall  sense  of  confusion  surrounding  post-modernism.  Added  to 
these  layers  of  complexity  are  various  types  of  post-modernism:  Skeptical  post-modernism  and  affirmative  post- 
modernism (Rosenau,  1995).  Skeptical  post-modernism  is  pessimistic,  negative  and  gloomy,  suggesting  “...  that  the 
post-modem  age  is  one  of  fragmentation,  disintegration,  malaise,  meaninglessness,  a vagueness  or  even  absence  of 
moral  parameters  and  societal  chaos”  (Rosenau,  1995,  p.  108).  Within  our  field,  Wilson  (1997a)  has  observed  that 
“...postmodern  thinking  can  lead  to  ...  positive  or  negative  outlooks  on  life”  (p.  303).  Although  affirmative  post- 
modernism tends  to  agree  with  the  skeptical  view  of  modernity  (Rosenau,  1995),  it  offers  “...a  more  hopeful, 
optimistic  view  of  the  post-modem  age”  (p.  108).  Rosenau  (1995)  posits  that  “many  affirmatives  argue  that  certain 
value  choices  are  superior  to  others,  a line  of  reasoning  that  would  incur  the  disapproval  of  the  skeptical  post- 
modernists” (p.  109).  Further,  there  are  extreme  and  moderate  versions  of  both  types  of  post-modernism  (Rosenau, 
1995). 

These  divergent  expositions  of  post-modernism  are  a major  source  of  confusion,  which  complicates  any 
attempts  at  definition.  This  paper  will  consider  various  explanations  of  post-modernism  and  then  suggest  a new 
definition  that  addresses  the  concerns  of  critics. 

Post-Modernism  as  a Philosophical  Orientation 

It  is  critically  important  to  distinguish  between  philosophy  and  theory  because  post-modernism  is  often 
described  as  a theory  (Yeaman,  1996)  when  it  is  actually  a philosophical  orientation  (Wilson,  1997a).  First, 
philosophy  is  interpreted  as  a composite  statement  of  beliefs  and  values  from  which  personal  purpose  and  direction 
are  derived  (Ely,  1970).  Second,  philosophy  is  a foundation  for  theory  (Koetting,  1996;  Smith  & Ragan,  1999; 
Snelbecker,  1974).  In  contrast,  a formal  theory  consists  of  hypotheses,  propositions,  and  laws  (Richey,  1986). 
Further,  a theory  is  an  organized  set  of  related  propositions  that  enable  people  to  explain,  predict  or  control  events 
(Hoover,  1995;  Richey,  1986). 
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Given  these  definitions,  post-modernism  is  a philosophical  orientation  because  it  does  not  possess  any 
predictive  power.  Similarly,  constructivism,  which  has  many  roots  in  post-modern  philosophies  (Wilson,  Teslow,  & 
Osman-Jouchoux,  1995),  is  often  viewed  as  an  educational  philosophy  because  it  does  not  have  the  explanatory 
power  of  psychological  learning  theories  (Lebow,  1995;  Smith  & Ragan,  1999). 

The  Role  of  Philosophy  in  Instructional  Technology  is  Ill-Defined 

Philosophy  is  important  in  our  field  because  it  serves  as  a foundation  for  theory  (Koetting,  1996;  Smith  & 
Ragan,  1999;  Snelbecker,  1974).  For  example,  Ertmer  and  Newby  (1993)  traced  the  origins  of  behaviorism, 
cognitivism  and  constructivism  back  to  their  philosophical  foundations  in  empiricism  and  rationalism.  The  roots  of 
any  science  can  be  traced  back  to  philosophical  origins  (Koetting,  1996;  Luiz,  1982;  Snelbecker,  1974). 

In  addition,  one’s  theoretical  framework  is  influenced  by  one’s  philosophical  orientation  to  the  world 
(Koetting,  1996).  For  example,  behavioral  theory  is  rooted  in  empiricism,  a philosophical  view  that  knowledge  is 
derived  from  experience  (Schunk,  1991).  Accordingly,  someone  who  accepts  the  notion  of  empiricism  may  be  more 
inclined  to  explain  learning  from  a behavioral  orientation.  While  it  may  seem  odd  that  behaviorism  can  be  discussed 
alongside  post-modernism,  the  point  to  be  made  is  that  one’s  philosophical  orientation  provides  insight  into  one’s 
values,  which  exert  influence  over  one’s  perceptions  of  the  relevance  of  research  (Richey,  1998).  Philosophical 
orientation  also  serve  as  a screening  device  that  mediates  decision-making  processes  (Luiz,  1982).  Recent 
perspectives  on  the  role  of  philosophy  suggest  that  applied  “fields  of  study,  such  as  instructional  design,  do  not  have 
educational  philosophies;  people  who  study  in  these  fields  do”  (Smith  & Ragan,  1999,  p.  14).  For  these  reasons,  it 
becomes  increasingly  important  to  explore  philosophy  because  instructional  technology  is  an  applied,  decision- 
oriented  field  (Reigeluth,  1983). 

Philosophy  is  important  for  other  reasons,  as  well.  Philosophical  inquiry  cultivates  the  intellectual  skills  of 
critical  thinking  and  offers  new  perspectives  on  solving  problems  (Morris,  1999).  James  D.  Finn  (1953/1996) 
believed  that  our  body  of  systematic  theory  needed  to  be  constantly  expanded  by  research  and  thinking  (emphasis 
added).  Finn  believed  in  vigorously  exploring  our  ideas  about  instructional  technology;  in  particular,  he  felt  that 
philosophizing  played  a critical  role  in  future  planning  (McBeath,  1972).  Ely  (1970)  felt  “...that  there  should  be  a 
philosophy  of  instructional  technology  and  that  it  should  vary  from  individual  to  individual”  (p.  81).  Still, 
philosophical  inquiry  has  been  relatively  absent  from  the  instructional  technology  literature  (Koetting  & 
Januszewski,  1991).  In  an  update  to  his  1970  article,  Toward  a Philosophy  of  Instructional  Technology , Ely  (1999) 
reminds  us  that  philosophies  change  in  response  to  social  conditions.  Ely  (1999)  suggests  that  “. . .if  there  is  any  new 
dimension  to  the  philosophy  held  by  many  professionals  in  the  field  it  is  change  itself  - the  readiness  to  consider, 
test  and  adopt  new  procedures  and  processes  in  the  goal  of  obtaining  more  efficient  and  effective  learning”  (Ely, 
1999).  As  with  Jim  Finn,  philosophy  plays  a critical  role  in  future  planning  for  Ely,  as  well. 

The  emergence  of  post-modernist  and  constructivist  orientations  in  the  field  are  beginning  to  spur  renewed 
interest  in  philosophical  inquiry.  In  1992,  Hylnka,  presented  a paper  titled  ‘Toward  a philosophy  of  educational 
technology”  to  the  AECT  Definitions  Committee  and  concluded  that  “any  philosophy  which  can  help  us  to 
illuminate  what  we  do,  how  we  do  it,  and  why  we  do  it,  is  worth  our  time  and  our  effort”  (Hlynka,  1992,  p.  4). 
Posing  questions  is  the  fundamental  task  of  philosophy,  which  is  the  basis  for  research  and  the  foundation  upon 
which  our  field  is  built  (Koetting,  1996). 

Part  Two:  Toward  a Definition  of  Post-Modernism 

Thus  far,  post-modernism  has  been  defined  as  a philosophical  orientation;  however,  its  relevance  to 
instructional  technology  requires  clarification.  In  an  effort  to  promote  clarity,  post-modernism  requires  further 
definition,  beyond  the  domain  of  philosophy.  An  interdisciplinary  review  of  the  literature  suggests  that  post- 
modernism is  approached  from  multiple  perspectives,  including  a general  social  condition  (Harvey,  1990;  Jencks, 
1992b;  Lyotard,  1979/1984),  an  intellectual  movement  (Doll,  1993)  and  as  an  historical  epoch  (Jencks,  1995). 
Anderson  (1995a)  believes  that  “...it’s  useful  to  make  a distinction  between  postmodemity  and  postmodernism  - the 
first  being  the  time  (or  condition)  in  which  we  find  ourselves,  the  second  being  the  various  schools  and  movements 
it  has  produced”  (pp.  6-7).  This  perspective  offers  guidance  for  constructing  a definition  of  post-modernism.  In 
order  to  promote  clarity,  this  paper  will  explore  post-modernism  as  both  a general  social  condition,  and  as  an 
intellectual  movement. 
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The  Post-Modern  Condition 

As  with  most  post-modern  concepts,  there  are  multiple  views  regarding  the  condition  of  postmodemity. 
Harvey  (1990)  proposes  that  postmodemity  is  a pervasive  condition  that  follows  the  breakdown  of  the 
“Enlightenment  project,”  a great  historical  era  where  rational  thought  and  the  scientific  method  replaced  superstition 
and  tradition.  According  to  Harvey  (1990),  “the  Enlightenment  project  ...  took  it  as  axiomatic  that  there  was  only 
one  possible  answer  to  any  question.  From  this  perspective  it  followed  that  the  world  could  be  controlled  and 
rationally  ordered  if  we  could  only  picture  and  represent  it  rightly”  (p.  27).  Many  definitions  of  the  post-modem 
condition  can  be  linked  to  the  French  philosopher,  Jean-Francois  Lyotard.  In  1979  Lyotard  authored  The 
Postmodern  Condition:  A Report  on  Knowledge,  upon  request  by  the  Council  of  Universities  of  the  Qufebec 
government.  In  Lyotard’s  (1979/1984)  report  on  the  state  of  knowledge  in  the  Western  world,  he  asserted  "...  that 
all  modem  systems  of  knowledge,  including  science,  had  been  supported  by  some  ‘metanarrative’  or  ‘grand 
discourse’  about  the  main  direction  of  history”  (Anderson,  1995a,  p.  4).  Anderson’s  explanation  is  among  the  most 
lucid:  “A  metanarrative  is  a story  of  mythic  proportions,  a story  big  enough  and  meaningful  enough  to  pull  together 
philosophy  and  research  and  politics  and  art,  relate  them  to  one  another,  and  - above  all  - give  them  a unifying 
sense  of  direction”  (p.  4).  Simply  put,  Lyotard’s  (1979/1984)  definition  of  postmodemity  is  “...incredulity  toward 
metanarratives”  (p.  xxiv).  Jencks  ( 1992a)  offers  another  view  that  contrasts  both  the  negative  and  positive 
attributes  of  the  post-modem  condition: 

The  increase  in  communication  (and  the  information  glut  and  advertisement),  the  growth  of 
knowledge  (and  the  consumer  society),  the  rise  of  leisure  (and  of  Disneyland  simulacra),  the 
flowering  of  Post-Fordism  (and  the  insecurity  of  workers),  the  emergence  of  a new  world  order 
(and  the  Pax  Americana ),  the  EC,  GATT  and  global  economy  (and  the  Third  World  debt  and  IMF 
riots)  - for  every  positive  post-modem  trend  there  is  a corresponding  negative  consequence,  (p. 

13) 

Perhaps  unknowingly,  Gustafson  (1995)  provided  one  of  the  best  descriptions  of  the  post-modem  condition,  as  it 
relates  specifically  to  the  field  of  IT: 

Rapid  changes  in  world  economic  conditions  are  creating  enormous  pressures  on  business  and 
industry  to  become  more  competitive  and  help  their  employees  become  more  productive.  Coupled 
with  these  pressures  are  the  demands  of  ever-changing  technology  and  the  information  explosion. 

In  the  past,  workers  could  “train  and  then  apply”  what  they  had  learned.  Today  the  question  is 
increasingly,  how  can  employees  “learn  while  applying?”  The  world  is  moving  so  fast  that 
knowledge  can  become  obsolete  even  before  it  can  be  analyzed,  developed  into  training  programs, 
and  offered  to  employees,  (p.  24) 

For  the  purpose  of  defining  post-modernism  as  a general  social  condition,  a synthesis  of  these  various  perspectives 
reveals  a set  of  common  themes:  globalization,  rapid  change  in  the  information  age  and  concern  for  (multicultural) 
people.  Simply  put,  the  post-modem  condition  refers  to  the  milieu  that  currently  exists  in  our  world,  today. 
Gustafson  offers  one  perspective  of  the  post-modem  condition  as  it  relates  to  instructional  technology.  Ely’s  (1999) 
concept  of  a “philosophy  of  change”  is;  yet,  another. 

The  Post-Modern  Intellectual  Movement 

The  section  that  follows  serves  to  illustrate  that  the  roots  of  post-modernism  can  be  traced  through  a variety 
of  disciplines.  In  order  to  improve  communication  in  instructional  technology  and  facilitate  theory  construction,  the 
elusive  language  of  post-modem  discourse  is  conspicuously  absent.  An  abridged  version  of  the  post-modem 
intellectual  movement  has  been  constructed  in  order  to  identify  the  sources  of  post-modem  thought.  The  author 
respectfully  acknowledges  that  each  of  the  foundations  of  the  post-modern  intellectual  movement  is  an  area  of 
inquiry  unto  itself:  (1)  structuralism,  (2)  semiotics,  (3)  poststructuralism,  (4)  deconstruction,  (5)  knowledge  and 
power,  (6)  critical  theory,  and  (7)  self-concept. 

Structuralism. 

Appignanesi  and  his  colleagues  (1995)  suggest  that  “postmodern  theory  is  a consequence  of  this  century’s 
obsession  with  language”  (p.  56).  Accordingly,  post-modernism  has  its  roots  in  structuralism  (Appignanesi  et  al., 
1995),  which  is  “...an  intellectual  movement  most  readily  associated  with  the  linguist  Ferdinand  de  Saussure  and  the 
anthropologist  Claude  Levi-Strauss”  (Bush,  1995,  p.  2).  Structuralism  asserts  that  the  meaning  of  language  (and 
culture)  can  be  derived  from  its  underlying  formal  systems  (Bush).  A central  premise  of  structuralism  was  the 
existence  of  “...a  systemic  ‘center’  that  organized  and  sustained  an  entire  structure”  (Bush  p.  2).  Structuralism  is 
concerned  with  "...  underlying  rules  and  conventions  that  enable  language  to  operate... the  social  and  collective 
dimension  of  language  [and]  the  infrastructure  of  language  common  to  all  speakers  on  an  unconscious  level” 
(Appignanesi  et  al.,  p.  57).  Saussure  also  proposed  that  within  a language  system,  the  signifier  (the  word)  carried 
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meaning,  and  the  signified  (the  concept)  was  that  to  which  it  refers  (Appignanesi  et  al.,  1995).  Together,  a signifier 
and  the  signified  are  a sign  and  meaning  is  derived  through  their  relationship,  which  is  socially  constructed. 

Scheel  and  Branch  (1993)  suggest  that  “...conversation  may  serve  as  a basis  for  designing  and 
implementing  culturally  pluralistic  instruction”  (p.  13)  because  the  goals  of  instruction  and  the  goals  of  conversation 
are  similar.  Both  attempt  to  focus  on  common  topics  and  create  shared  meaning,  which  address  some  of  the 
concerns  of  structuralism. 

Semiotics . 

Semiotics  is  the  science  of  signs  and  sign  systems  and  their  relationships  to  language  and  human  behavior 
(Morris,  1946).  Further  insight  into  semiotics  is  offered  by  Appignanesi  and  his  colleagues  (1995): 

Saussure  opened  the  way  to  analyzing  culture  itself  as  a system  of  signs  by  proposing  that  structural 
linguistics  was  part  of  semiology,  a general  science  of  signs  which  studies  the  various  systems  of  cultural 
conventions  which  enable  human  actions  to  signify  meaning  and  hence  become  signs. 

Semiotics  is  clearly  rooted  in  structuralism,  which  declares  that  meaning  is  founded  on  a system  of  shared 
conventions  (or  is  socially  constructed).  While  the  obvious  application  of  semiotics  in  our  field  would  address 
message  design,  a body  of  literature  in  the  humanities  offers  unique  perspectives  on  the  implications  of  post-modern 
thought  in  instructional  technology.  In  particular,  Jay  David  Bolter  (1991)  has  explored  the  notion  of  electronic 
writing  spaces,  claiming  that  the  computer  is  a self-contained  system  where  the  entire  process  of  semiosis  occurs. 
Bolter  reminds  us  that  in  electronic  spaces,  people  learn  to  read  and  write  differently,  interacting  with  text  that  is 
now  a texture  of  signs  that  include  words  and  graphic  elements.  Michael  Joyce  (1995),  an  acclaimed  hypertext 
novelist,  explores  technology  in  relation  to  new  forms  of  art,  instruction,  teaching  and  writing.  Joyce  addresses  the 
nomadic  movement  of  ideas,  enabled  through  the  electronic  medium,  and  he  challenges  current  conceptions  of 
human  capabilities.  These  ideas  have  implications  for  the  various  ways  in  which  meaning  can  be  constructed  and/or 
facilitated  through  technology  and  instructional  design  processes. 

Poststructuralism. 

Poststructuralism  attacks  the  structuralist  notion  of  a systemic  “center”  - the  underlying  rules  of  a social 
system  - that  organizes  and  sustains  language  and  culture  (Bush,  1995).  Poststructuralist  critics  argue  that  language 
lacks  a central  organizing  structure,  and  is  inherently  unstable  and  ambiguous  (Bush,  1995).  Bush  (1995)  suggests 
that  poststructuralism  generally  includes  three  main  features:  (1)  The  primacy  of  theory,  (2)  the  decentering  of  the 
subject,  and  (3)  the  fundamental  importance  of  the  reader.  These  features  are  explained  in  the  following  paragraphs. 

The  primacy  of  theory.  Bush  (1995)  suggests  that  poststructural  criticism  is  laden  with  theory,  the  nature  of 
which  challenges  and  subverts  the  enduring  assumptions  and  beliefs  of  western  civilization.  As  a consequence, 
“...poststructural  criticism  has  been  associated  with  an  adversarial  stance  that  often  takes  on  the  established 
institutional  and  political  forces  in  American  society”  (Bush,  1995,  p.  2). 

The  decentering  of  the  subject.  Contrary  to  humanistic  ideologies,  "...  the  poststructural  subject  or  self  is 
seen  to  be  incoherent,  disunified,  and  in  effect  ‘decentered  ...’  (Bush,  1995,  p.  2).  Steiner  Kvale  (1995)  offers 
further  insight: 

The  focus  on  language  implies  a decentralization  of  the  subject.  The  self  no  longer  uses  language 

to  express  itself;  rather  the  language  speaks  through  the  person.  The  individual  self  becomes  a 

medium  for  the  culture  and  its  language  (p.  22). 

People  are  seen  as  “commentators”  who  convey  ‘"unconscious  mainstream  ideologies”  or  as  transmitters  of  “various 
cultural  constructs  ...  created  by  the  structures  of  power  in  a given  social  environment...”  (Bush,  1995,  p.  2). 

The  fundamental  importance  of  the  reader.  Bush  (1995)  states  that  “with  the  destabilizing  or  decentering 
of  the  author  and  in  more  general  terms  of  language  as  a system,  the  reader  or  interpreter  has  become  the  focal  point 
of  much  more  poststructural  theorizing”  (p.  2).  Essentially,  this  tenet  posits  that  readers  create  their  own  meanings, 
regardless  of  the  author’s  intentions. 

Poststructural  thinking  in  instructional  technology  would  encourage  an  examination  of  the  relationships 
among  the  designer/developer  (author),  the  learner  (reader)  and  the  use  of  language  in  instructional  systems.  This 
type  of  inquiry,  like  Rowland’s  (1993)  concept  of  “reflection-in-action,”  views  design  as  a reflective  conversation 
with  the  materials  of  the  situation  (Schon,  as  cited  in  Rowland,  1993).  The  goal  of  this  type  of  intelligent  activity  is 
not  a fixed  understanding;  but,  rather  a more  integrative  awareness  of  the  various  ways  in  which  instructional 
technology  can  be  used  to  help  learners  create  their  own  meanings  or  facilitate  meaning  that  is  more  closely  aligned 
with  the  designer/developer’s  intentions.  Discourse  analysis  is  another  poststructural  technique  for  revealing 
systems  of  thought  that  "...  operate  at  a linguistic  level  to  produce  and  regulate  knowledge”  (De  Vaney  & Butler, 
1996,  p.  5).  In  their  examination  of  early  educational  technology  texts,  De  Vaney  and  Butler  (1996)  employed  a 
rhetorical  technique  that  explored  the  hierarchy  of  topics  and  subject  matter  as  represented  in  the  table  of  contents 
and  indices.  In  addition,  the  authors’  intentions  were  investigated  in  prefaces  and  forewards;  and,  these  findings 


were  then  related  to  discourse-specific  systems  of  thought.  The  value  of  this  mode  of  inquiry  was  an  illustration  of 
the  systems  of  thought  that  have  shaped  the  production  of  knowledge  in  our  field  and  the  influences  that  these 
discourses  continue  to  exert  in  instructional  technology,  today. 

Deconstruction. 

Deconstruction  is  an  offshoot  of  poststructural  theory  that  was  introduced  by  the  philosopher  Jacques 
Derrida.  Hlynka  and  Yeaman  (1992)  offer  a summary  of  the  basic  tenets  of  deconstruction  in  their  ERIC  Digest 
titled  Postmodern  Educational  Psychology: 

1.  Consider  concepts,  ideas  and  objects  as  texts.  Textual  meanings  are  open  to  interpretation. 

2.  Look  for  binary  oppositions  in  those  texts.  Some  usual  oppositions  are  good/bad,  progress/tradition, 
science/myth,  love/hate,  man/woman,  and  truth/fiction. 

3.  “Deconstruct”  the  text  by  showing  how  the  oppositions  are  not  necessarily  true. 

4.  Identify  texts  which  are  absent,  groups  who  are  not  represented  and  omissions,  which  may  or  may  not 
be  deliberate,  but  are  important  (pp.  1-2). 

One  of  the  assumptions  of  deconstruction  is  that  virtually  any  facet  of  cultural  life  can  be  interpreted  as  a text  and 
subsequently  deconstructed  (Harvey,  1990).  In  instructional  technology,  deconstruction  can  shed  light  upon  the 
process  of  interpretation  and  lend  support  to  the  constructivist  notion  that  learners  construct  knowledge  rather  than 
acquire  it.  By  reducing  texts  to  a play  of  signs,  the  goal  of  deconstruction  is  to  show  that  perfect  signification 
between  a sign  and  its  referent  can  never  exist;  thus,  an  insistence  on  the  endless  quality  of  interpretation  (Bolter, 
1991). 

Knowledge  and  Power. 

The  French  philosopher,  Michel  Foucault  is  the  postmodern  theorist  recognized  for  addressing  the 
concerns  of  power  and  legitimation  (Appignanesi  et  al.,  1995).  Appignanesi  and  his  colleagues  provide  a concise 
summary  of  Foucault’s  perspective:  “he  tackles  power  from  the  unusual  angle  of  knowledge  as  systems  of  thought 
which  become  controlling,  that  is,  socially  legitimated  and  institutional”  (p.  82).  Lyotard  (1979/1984)  has  expressed 
similar  views;  he  presents  knowledge  as  a commodity,  he  questions  who  will  have  access  to  it,  and,  he  raises 
questions  about  its  legitimation. 

Knowledge  is  seen  as  the  most  important  resource  and  learning  the  most  important  capability  for  businesses  today 
(Zack,  1999).  Knowledge  management,  which  is  concerned  with  “...recognizing,  documenting,  and  distributing 
explicit  and  tacit  knowledge  ...”  (Rossett,  1999,  p.  64),  can  be  linked  to  analysis  activities  in  instructional 
technology.  While  knowledge  management  isn’t  intended  to  replace  training,  Rossett  (1999)  reminds  us  that  we 
have  access  to  more  information  than  ever  before  and  our  challenge  as  instructional  technologists  is  integrative.  Not 
only  should  instruction  incorporate  data  from  diverse  knowledge  bases,  it  should  also  complement  intellectual 
capital. 

Critical  Theory. 

Critical  Theory  emerged  out  of  the  Frankfurt  School,  a German  philosophical  and  sociological  movement 
that  generally  believes  that  scientific  inquiry  is  riddled  with  non-theoretical  interests  because  theory  development  is 
a product  of  social  processes  (Honderich,  1995).  Critical  theory  promotes  a radical  change  in  theory  and  practice, 
encouraging  that  every  one-sided  doctrine  should  be  subjected  to  criticism.  In  his  book,  Knowledge  and  Human 
Interests , Habermas  (1971)  advanced  the  notion  of  critical  theory,  which  situates  knowledge  within  a philosophical 
framework  based  upon  three  forms  of  valid  inquiry,  which  produce  three  forms  of  valid  knowledge  (information, 
interpretations  and  analyses).  Just  as  post-modernism  means  different  things  to  different  people,  alternative 
definitions  of  critical  theory  have  emerged  in  the  literature.  Anderson  (1995a)  states  that  “in  literature  and  the  arts, 
we  have  critical  theorists  who  insist  that  when  you  experience  a work  of  art  you  don’t  simply  take  in  the  artist’s 
intention,  but  actively  participate  in  creating  whatever  meaning  or  message  you  find”  (p.  9). 

In  1983,  Koetting  (1983)  explored  the  notion  of  knowledge  in  instructional  technology  and  developed  an 
epistemological  framework  for  inquiry  in  our  field.  Koetting’s  paper,  “Philosophical  Foundations  of  Instructional 
Technology,”  (1983)  discussed  the  implications  for  future  research,  and,  proposed  alternative  philosophical  and 
theoretical  frameworks  for  inquiry  within  the  field.  While  an  exploration  of  Koetting’s  (1983)  work  is  beyond  the 
scope  of  this  paper,  the  point  is  that  “...  the  field  should  embrace  a wide  variety  of  research  paradigms  ...”  (Driscoll, 
1991,  p.  310).  Koetting’s  (1983)  paper  may  be  one  of  the  first  works  to  directly  relate  critical  theory  to  the  field  of 
instructional  technology  (Nichols  & Allen-Brown,  1996). 

Self  concept. 

Jacques  Lacan,  the  French  psychoanalyst,  proclaimed  that  the  unconscious  is  structured  as  a language,  and 
that  thinking  was  dependent  upon  language  (Appignanesi  et  al.,  1995).  With  its  roots  in  psychoanalytic  theory,  post- 
modern assumptions  about  self  concept  posit  that  individual  identity  is  “...  constructed  (and  frequently 
reconstructed)  out  of  many  cultural  sources”  (Anderson,  1995a,  p.  10). 
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The  natural  connection  between  self  concept  and  instructional  technology  would  be  a refined  exploration  of 
learner  characteristics.  Anderson  (1995a)  posits  that 

...the  postmodern  individual  is  a member  of  many  communities  and  networks,  a participant  in 

many  discourses,  an  audience  to  messages  from  everybody  and  everywhere  - messages  that 

present  conflicting  ideals  and  norms  and  images  of  the  world  (p.  9). 

This  notion  suggests  the  importance  of  considering  more  variables  that  could  potentially  affect  learning  and 
instruction.  The  implication  is  that  knowledge  of  the  “many  communities  and  networks”  to  which  learners  belong, 
may  yield  strategies  for  improving  the  efficacy  of  instruction. 

General  Assumptions  About  Post-Modernism 

Post-modernism  has  been  defined  as  a philosophical  orientation,  a social  condition  and  as  an  intellectual 
movement.  In  support  of  the  latter  definition,  the  foundations  of  the  intellectual  movement  were  outlined;  however, 
the  definition  is  still  incomplete.  Doll  (1993)  cautions  that  "...  it  is  impossible  to  give  one  overarching  definition  of 
post-modernism:  The  movement  is  too  new  to  define  itself  and  too  varied  and  dichotomous  for  any  one  branch  to  be 
representative”  (p.  5).  In  response  to  the  question:  “What  is  post-modernism?”,  Jencks  (1995)  proclaims  that  “...its 
continual  growth  and  movement  mean  that  no  definitive  answer  is  possible  - at  least  not  until  it  stops  moving”  (p. 
29). 

Given  these  challenges,  I have  constructed  eight  general  assumptions  about  post-modernism  from  an 
interdisciplinary  review  of  the  literature.  The  assumptions  about  post-modernism  follow  for  consideration, 
discussion,  and/or  adoption.  Clearly,  this  is  not  an  exhaustive  list  and  many  more  assumptions  about  post- 
modernism are  sure  to  exist.  The  underlying  purpose  for  constructing  these  assumptions  is  to  synthesize  a large 
body  of  information  and  summarize  the  post-modern  ideology. 

Post-Modern  Assumptions 

1.  Pluralism . An  essential  goal  of  post-modernism  is  to  promote  pluralism  (Jencks,  1992a).  Jencks 
believes  that  “...pluralism  is  the  leading  ‘ism’  of  post-modernity  ...  [and  it]  means  the  end  of  a single  world  view 
and,  by  extension,  a ‘war  on  totality’,  a resistance  to  single  explanations,  a respect  for  difference  and  a celebration  of 
the  regional,  local  and  particular”  (p.  11).  The  concept  of  pluralism  is  pervasive  throughout  the  post-modern 
intellectual  movement  as  evidenced  by  Hlynka  and  Yeaman’s  (1992)  proclamation  that  “...if  there  are  multiple 
ways  of  knowing  then  there  must  be  multiple  truths”  (p.  3). 

Pluralism  can  be  seen  as  an  underlying  philosophy  in  instructional  technology  that  allows  various 
perspectives  to  co-exist.  For  example,  a post-modern  worldview  would  suggest  that  researchers  should  not  yield  to 
dominant  research  paradigms.  Practitioners  would  celebrate  a type  of  theoretical  pluralism  where  traditional  and 
non-traditional  approaches  to  learning  and  instruction  could  complement  each  other. 

2.  Eclecticism.  An  essential  style  of  post-modernism  is  eclecticism  (Jencks,  1992b).  According  to  Jencks 
(1995),  “post-modernism  is  fundamentally  the  eclectic  mixture  of  any  tradition  with  that  of  the  immediate  past”  (p. 
27).  According  to  Harvey  (1990),  “Derrida  considers  ...  collage/montage  as  the  primary  form  of  postmodern 
discourse  [i.e.,  painting,  writing,  architecture]”  (p,  51)  because  it  produces  a signification  that  is  “neither  univocal 
nor  stable.” 

As  a style,  eclecticism  shares  the  pluralistic  philosophy  described  above;  however,  in  instructional 
technology  it  might  become  manifested  as  the  combination  of  models  to  produce  viable  instruction  (Richey,  1995). 
Instructional  technology  is  an  eclectic  field,  in  and  of  itself,  where  ideas  and  resources  from  other  disciplines 
become  integrated  into  our  theoretical  and  practical  bases  of  knowledge. 

3.  Knowledge.  An  essential  tenet  of  post-modernism  is  that  people  construct  knowledge  (Wilson  et  al., 
1995).  Although  the  concept  that  knowledge  is  constructed  rather  than  acquired  is  not  necessarily  a post-modern 
invention,  it  clearly  positions  the  post-modern  stance  on  the  constructivist  side  of  the  fence.  As  previously  discussed, 
constructivism  has  many  roots  in  post-modern  philosophies  (Wilson  et  al.,  1995).  While  this  assumption  about 
knowledge  is  seemingly  innocuous,  it  is  laden  with  several  post-modern  issues.  From  Lyotard  (1979/1984),  one  of 
the  defining  characteristics  of  post-modernism  is  a disbelief  in  metanarratives,  or  established  knowledge  (a  concept 
that  is  also  linked  to  pluralism  because  it  rejects  single  explanations).  Lyotard  (1979/1984)  also  promoted  the  idea 
that  “knowledge  is  and  will  be  produced  in  order  to  be  sold,  it  is  and  will  be  consumed  in  order  to  be  valorized  in  a 
new  production:  in  both  cases,  the  goal  is  exchange”  (p.  4).  Lyotard’s  perspective  appears  visionary  in  light  of 
contemporary  issues  surrounding  knowledge  management.  Yeaman  (1996)  believes  that  “separating  the  author 
from  the  authority  of  a text  requires  acknowledging  the  political  issues  of  knowledge  and  power”  (p.  276),  an  idea 
that  is  related  to  post-modern  assumptions  about  truth. 
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4 . Truth.  An  essential  opinion  of  post-modern  thinking  is  that  truth  is  grounded  in  subjective  experience 
(Wilson  et  al.,  1995).  In  addressing  the  question  of  where  truth  lies,  Hlynka  (1996)  explains  that  “traditionally,  one 
assumes  that  the  author  of  a work  is  the  ultimate  authority  ...  [however]  ‘truth  now  lies  in  the  text  itself,  while  the 
new  task  becomes  one  of  interpretation”  (p.  256).  Further,  Hlynka  illustrates  how  contemporary  literary  theory  and 
reader  response  theory  suggest  that  “...authority  lies  not  only  in  the  author  who  wrote  it,  or  in  the  text  that  says  it, 
but  in  the  reader  who  reads  it”  (p.  256).  The  French  sociologist,  Jean  Baudrillard,  advanced  the  notion  of  the 
simulacrum,  which  occurs  when  the  distinction  between  representation  (e.g.,  art;  signs)  and  reality  breaks  down, 
marking  the  absence  of  a basic  reality  (Appignanesi  et  al.,  1995).  Baudrillard’s  conclusion  relates  to  the  notion  of 
truth  (albeit  an  extreme  form  of  postmodernism);  and,  his  ideas  remain  an  important  part  of  the  movement’s 
intellectual  history.  A more  common  opinion  is  offered  by  the  philosopher,  Richard  Rorty  (as  cited  in  Anderson, 
1995a),  who  posits  that  “truth  is  made  rather  than  found  (p.  8);”  a view  that  interprets  reality  as  being  socially 
constructed. 

As  with  the  assumption  about  knowledge,  these  ideas  about  the  nature  of  truth  are  reminiscent  of 
constructivist  ideas  in  instructional  technology.  In  particular,  meaning  is  understood  to  be  rooted  in  experience, 
which  supports  the  argument  for  situating  learning  activities  in  authentic  environments  (Duffy  & Jonassen,  1992). 
In  this  regard,  authority  or  truth  would  reside  not  only  in  the  learning  environment  and  materials,  but  also  with  the 
learner. 

5.  Language.  An  essential  theme  of  postmodern  thought  is  that  language  is  deeply  involved  in  the  social 
construction  of  reality  (Anderson,  1995a).  From  a poststructuralist  perspective,  Weedon  (as  cited  in  Anderson  & 
Damarin,  1996)  suggests  that  “language  enables  people  to  think,  speak  and  give  meaning  to  the  world  around  them” 
(p.  270).  Further  clarification  about  language  is  offered  by  Anderson  and  Damarin  (1996)  who  state  that  “how 
people  write,  talk,  and  otherwise  communicate  about  what  they  know,  do,  and  believe  reflects  the  ways  they  are 
shaped  by  particular  discourse  communities”  (p.  270).  An  underlying,  poststructural  issue  associated  with  the  role 
of  language  is  that  meaning  is  indeterminate  because  language  is  inherently  unstable  and  ambiguous  (Bush,  1995). 

Scheel  and  Branch  (1993)  have  reviewed  the  benefits  of  focusing  on  common  topics  and  creating  shared 
meaning  through  dialogue.  Another  concept  of  learning  is  based  upon  hermeneutic  principles  that  views  learning  as 
a process  of  interpretation  (Jonassen  et  al.,  1997).  While  the  instructional  designer  can  use  language  that  increases 
the  probability  of  “correct”  interpretation,  Jonassen  and  his  colleagues  (1997)  also  explain  how  hermeneutics  can  be 
used  in  instructional  design  to  assist  with  personal  interpretation.  Although  seemingly  simplistic,  discussion 
questions  at  the  beginning  of  a lesson  can  be  used  purposefully  to  stimulate  the  development  of  subjective  meaning 
(Jonassen  et  al.,  1997). 

6.  Communication.  An  essential  issue  in  post-modernism  is  communication  (Harvey,  1990).  Transmission 
models  of  communication  typically  include  a sender  who  encodes  a message,  the  message,  a channel;  and,  a receiver 
who  decodes  information  (Shannon  & Weaver,  1949).  Additional  components  of  the  model  usually  include  noise 
and  feedback.  Communication  is  a foundational  issue  in  post-modernism  because  each  component  (and  every 
element)  in  the  communications  process  is  scrutinized.  Post-modern  assumptions,  as  presented  here,  posit  that 
authority  lies  not  only  in  the  sender  and  the  message,  but  also  in  the  receiver.  They  also  address  post-modern 
questions  of  knowledge,  power  and  legitimation  as  they  relate  to  the  information  source  and  the  message.  Finally, 
with  respect  to  noise  and  feedback,  the  post-modern  concerns  about  communication  are  also  entwined  with  post- 
modern sciences  of  complexity,  described  below. 

7.  Complexity.  Jencks  (1992a)  describes  a post-modern  science  as  shifting  “. . . from  fairly  inanimate  matter 
(planets  and  physical  objects)  to  living  systems  (social  groups  as  well  as  individuals)  (p.  15).”  As  an  example, 
Jencks  (1992a)  points  to  a view  of  nature  as  a self-organizing  system.  From  this  perspective,  systems  are  interpreted 
as  dynamic,  living  entities  which  consist  of  non-linear  processes  and  a high  degree  of  feedback,  which  characterize 
all  life  (Jencks,  1992a).  An  applied  example  of  this  concept  could  be  Artificial  Intelligence,  complex  computer 
systems  that  are  designed  to  adapt  to  environmental  stimuli.  Chaos  methodology  is  another  application  that  “... 
shifts  emphasis  from  relationships  of  cause  and  effect  to  more  interactive,  multivariant  approaches  that  stress  the 
importance  of  defining  patterns,  form,  self-organization,  and  adaptive  qualities  of  complex  processes”  (Krippner  & 
Winkler,  1995,  p.  166). 

A post-modern  interpretation  of  chaos  theory  would  jettison  the  notion  of  deterministic  predictability  and 
posit  that  chaos  is  inherent  in  all  systems;  thus,  explaining  the  irregular  behavior  of  nonlinear  dynamic  systems.  The 
implications  of  this  orientation  would  be  an  examination  of  the  myriad  variables  that  interact  to  produce  learning 
and  “developing  metacognitive  awareness  in  learners  as  a way  of  helping  them  deal  with  a complex  and  ill- 
structured  world”  (Jonassen  et  al.,  1 997,  p.  32).  There  may  also  be  implications  for  understanding  the  role  of  human 
emotion  as  a strategy  for  managing  chaos  in  learning,  as  well  (Jonassen  et  al.,  1997). 
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8.  Self  Post-modernism  interprets  an  individual  as  a pluralistic  person  with  a disappearing  self.  Anderson 
(1995b)  states  that  “the  postmodern  person  is  a multi-community  person,  and  his  or  her  life  as  a social  being  is  based 
on  adjusting  to  shifting  contexts  and  being  true  to  divergent  - and  occasionally  conflicting  - commitments”  (p.  128). 
Further,  Anderson  (1995b)  posits  that  “individuals  negotiate  (and  renegotiate)  personal  identity,  struggling  to  make 
internal  peace  among  the  multiple  components  of  their  selves  and  the  claims  of  the  different  communities  to  which 
they  are  connected”  (p.  128).  Gergen,  (1995)  a leading  exponent  of  post-modern  psychology,  challenges  traditional 
assumptions  about  human  development  as  “...a  single,  basic  self  to  which  we  can  be  true”  (p.  137)  in  favor  of  the 
idea  that  people  carry  the  potential  for  many  selves  which  can  be  realized  in  various  social  settings  (Anderson, 
1995b). 

These  ideas  are  opposed  to  beliefs  about  the  importance  of  establishing  a strong,  integrated  sense  of 
personal  identity,  which  paints  a one-dimensional  picture  of  the  learner.  While  there  may  be  central  tendencies 
related  to  the  concept  of  self,  post-modernists  would  argue  that  we  have  paid  to  much  attention  to  them  and  have 
ignored  the  complexity  of  human  existence  (Gergen,  1995).  In  our  field,  the  concept  of  self  reminds  us  that 
instructional  design  needs  to  address  more  than  practical  issues  and  provide  for  the  human  spirit  (Keller,  1979) 

Part  Three:  Potential  Contributions  of  Post-Modern  Concepts  in  Instructional  Technology 

The  eight  general  assumptions  just  reviewed  demonstrate  the  depth  and  range  of  post-modern  thought. 
Together,  they  represent  a body  of  ideas  that  could  define  a post-modern  philosophical  orientation;  however,  these 
assumptions  are  still  difficult  to  relate  to  the  field  of  instructional  technology.  Accordingly,  the  following  section 
will  summarize  several  core  concepts  about  post-modernism,  from  which  potential  contributions  to  the  field  of 
instructional  technology  can  be  discussed.  The  core  concepts  that  define  a post-modern  philosophy  of  instructional 
technology  include: 

1.  The  philosophical  “core”  of  post-modern  instructional  technology  is  a belief  in  pluralism,  which  can  be 
described  as  respect  for  difference  and  a resistance  to  single  explanations. 

2.  Knowledge,  truth  and  reality  are  constructed  by  people  and  groups  of  people. 

3.  Criticism  is  an  appropriate  method  for  inquiry  in  instructional  technology. 

4.  Systems  are  interpreted  as  highly  complex  entities  with  adaptive  qualities. 

The  philosophical  “core”  of  post-modern  instructional  technology  is  a belief  in  pluralism , which  can  be 
described  as  respect  for  difference  and  a resistance  to  single  explanations.  As  a defining  characteristic  of  post- 
modernism, pluralism  is  a concept  that  has  pervasive  and  far-reaching  impact  in  instructional  technology.  At  the 
surface,  pluralism  means  that  multiple  perspectives  may  be  valid  and  that  there  is  no  single  best  way  to  develop 
instruction  (Davies,  1981/1991;  Hlynka  & Yeaman,  1992).  Similarly,  pluralism  in  our  field  would  posit  that  there  is 
no  single  best  model  or  theory  of  learning.  From  an  applied  perspective,  pluralism  could  be  seen  as  an  eclectic 
approach  where  multiple  views  of  content,  strategies  and  perspectives  are  offered  (Wilson  et  al.,  1995);  however, 
this  is  still  a surface  view  of  the  concept. 

Beneath  the  surface;  however,  the  concept  of  pluralism  runs  deeper.  The  issues  of  integration  that  affect 
the  field  (Seels,  1995)  could  be  guided  by  post-modern  philosophy,  particularly  with  respect  to  “the  integration  of 
knowledge  from  the  arts  with  knowledge  from  the  sciences  in  an  age  of  technology”  (p.  252).  Post-modernism 
accepts  the  idea  that  art,  craft  and  science  could  co-exist  in  instructional  technology.  While  diverse  views  of  the  art, 
craft  and  science  debate  continue  to  emerge  in  the  literature  (see  Clark  & Estes,  1998;  Braden,  1996;  Merrill,  Drake, 
Lacy,  Pratt,  & ID2  Research  Group,  1996;  Davies,  1981/1991),  post-modernism  celebrates  these  multiple 
perspectives. 

The  implications  of  pluralism  in  relation  to  research  and  the  needs  of  practitioners  are  also  far-reaching. 
For  example,  the  notion  of  the  learner  as  a pluralistic  person,  situated  in  contexts  that  are  continuously  shifting, 
suggests  the  need  to  consider  a multiplicity  of  variables  related  to  learner  characteristics  and  ecological  systems  (see 
Bronfenbrenner,  1979;  1988),  all  of  which  influence  learning  and  instruction.  Another  example  concerns  alternative 
research  methods  that  are  emerging  in  the  literature  including  multiple  interpretations  of  data  and  self-reflexive 
technique,  the  recognition  of  one’s  own  values  in  the  research  process  (Anderson  & Damarin,  1996).  These 
examples  provide  a framework  for  conceptualizing  the  implications  of  pluralism  in  instructional  technology.  The 
fundamental  concept  of  pluralism  is  that  multiple  perspectives  are  valid,  an  ideology  that  can  be  applied  to  various 
facets  of  the  theory  and  practice  of  instructional  technology. 

Knowledge,  truth  and  reality  are  constructed  by  people  and  groups  of  people.  As  a foundation  for 
constructivism  (Wilson  et  al.,  1995),  post-modern  philosophies  of  knowledge,  truth  and  reality  are  exerting  great 
influence  on  the  field.  Post-modernists  believe  that  knowledge  is  something  that  is  created  rather  than  found,  which 
is  mirrored  in  constructivist  ideas  about  cognition  (i.e.,  people  create  meaning  through  experience  as  opposed  to 
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acquiring  it).  The  concepts  of  subjective  truth  and  socially  constructed  reality  are  also  beliefs  that  constructivism 
and  post-modernism  share.  Together,  the  concepts  of  knowledge,  truth  and  reality  have  helped  to  mold  and  shape 
the  emerging  theory  base  for  constructivism,  including  agenda-building  around  specific  issues  such  as  situated  and 
collaborative  learning . 

The  philosophical  view  that  reality  is  socially  constructed  can  provide  a framework  for  addressing  issues 
surrounding  cultural  diversity.  For  example,  Scheel  and  Branch  (1993)  have  offered  strategies  for  designing 
culturally  pluralistic  instruction  based  upon  the  premise  that  one’s  cultural  background  will  influence  interactions 
between  the  learner,  content,  teacher,  media  and  context.  Recognizing  that  learners  are  culturally  inscribed  and 
relational  to  the  societal  context,  they  believe  that  language  and  conversation  can  be  used  strategically  to  promote 
cultural  pluralism  and  increase  the  potential  for  learner  achievement.  These  ideas  can  be  directly  linked  to  diverse 
theories  of  language  that  provide  a foundation  for  the  post-modern  intellectual  movement  discussed  earlier. 

Criticism  is  an  appropriate  method  for  inquiry  in  instructional  technology.  Scientific  and  critical 
perspectives  can  be  complementary  in  the  field  of  instructional  technology  (Wilson,  1997b).  For  example,  a 
scientific  approach  in  our  field  would  pose  research  questions  that  require  predictions  and  hypotheses  while  a 
critical  perspective  is  more  like  literary  or  film  criticism,  exploring  various  layers  of  meaning.  These  approaches 
can  be  integrated  and  work  together  because, 

Science  and  technology,  if  not  checked,  would  tend  to  see  things  in  terms  of  their  instrumental 
value,  in  terms  of  their  scientific  classifications.  Criticism,  most  skillfully  practiced  by  the 
postmodern  theorists,  brings  balance  to  the  picture  by  closely  examining  the  details.  Technology 
alone  tends  to  be  problem-driven  and  goal -based.  Most  attention  goes  to  whether  objectives  are 
achieved.  Criticism  looks  beyond  the  objectives  to  examine  the  unintended  side  effects,  the 
secondary  meanings,  the  shades  of  gray  (Wilson,  1997b,  p.  25). 

While  criticism  should  not  become  an  exclusive  method  of  inquiry,  its  value  in  instructional  technology  is  still 
misunderstood.  The  rationale  for  criticism  is  summarized  by  Belland  and  his  colleagues  (1991):  “as  a field  which 
draws  knowledge  and  practice  from  a wide  range  of  arts  and  sciences,  educational  technology  should  be  able  to  use 
a variety  of  ways  of  investigating  and  knowing  in  order  to  guide  inquiry  and  practice”  (p.  151). 

Systems  are  interpreted  as  highly  complex  entities  with  adaptive  qualities.  While  the  humanities  provide  a 
major  source  of  post-modern  thought  (Hlynka  & Yeaman,  1992),  post-modern  science  interprets  systems  as  highly 
complex  entities  with  adaptive  qualities  (Jencks,  1992a).  Although  the  growing  base  of  literature  that  addresses  post- 
modern instructional  technology  has  not  yet  explicated  the  implications  of  post-modern  science,  the  concept  of 
highly  complex  systems  has  received  some  attention  in  the  field  (see  You,  1993).  As  learning  and  education 
become  increasingly  more  complex  in  our  knowledge-based  society  (Trilling  & Hood,  1999)  the  concept  of  complex 
systems  could  offer  a philosophical  perspective  to  guide  future  inquiry.  For  example,  the  post-modern  concept  of 
complexity  could  offer  insight  into  research  on  flexible  learning  systems  that  keep  pace  with  rapid  changes  in 
organizational  structures,  processes  and  information  technology  (a  by-product  of  the  post-modern  condition 
discussed  earlier).  Accordingly,  a post-modern  philosophical  orientation  might  support  the  need  for  contextual 
analysis  which  broadens  the  number  of  variables  that  designers  consider  during  the  design  of  instructional  systems 
(Tessmer  & Richey,  1997). 

Conclusion 

This  paper  has  attempted  to  illustrate  the  relevance  of  post-modernism  in  the  field  of  Instructional 
Technology.  In  pursuit  of  this  goal,  post-modernism  was  defined  as  a philosophy,  social  condition  and  an 
intellectual  movement.  As  a philosophical  orientation,  post-modernism  values  multiple  perspectives  and  the 
contextual  construction  of  meaning  (Wilson,  1997a).  The  post-modern  condition  describes  a rich  and  diverse  milieu 
that  is  influenced  by  globalization,  rapid  change  in  the  information  age  and  multiculturalism.  As  an  intellectual 
movement,  post-modernism  is  deeply  rooted  in  theories  of  language  and  various  disciplines  including  the  arts, 
sciences  and  humanities.  As  an  eclectic  field,  instructional  technology  integrates  ideas  from  a variety  of  disciplines, 
which  may  help  to  explain  the  emergence  of  post-modern  thought  within  the  discipline.  Seels  (1989)  describes  the 
field  as  the  confluence  of  three  disciplines,  including  media  in  education,  psychology  of  instruction  and  systematic 
approaches  to  education.  Richey  (1986)  has  outlined  four  theoretical  foundations  of  the  field:  communications 
theory,  general  systems  theory,  learning  theory  and  instructional/curriculum  theory.  Given  these  broad  perspectives 
on  the  origins  of  instructional  technology,  the  roots  of  post-modernism  (e.g.,  semiotics,  psychology,  theories  of 
language)  can  be  linked  to  our  history  and  should  not  be  ignored. 

The  voices  of  our  founders  and  the  early  discourses  of  the  field  have  established  a valuable  role  for 
philosophical  inquiry  in  instructional  technology;  however,  the  path  was  never  very  clear.  And,  with  post- 
modernism, there  are  multiple  paths  that  diffuse  meaning.  Based  upon  an  interdisciplinary  review  of  the  literature, 
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this  paper  has  explored  the  sources  of  confusion  surrounding  post-modernism.  Next,  three  definitions  of  post- 
modernism were  proposed  (i.e.,  a philosophy,  a social  condition  and  an  intellectual  movement).  Then,  the  potential 
contributions  for  the  field  were  discussed,  all  in  order  to  promote  communication  within  the  field  and  facilitate  an 
agenda-building  process  around  post-modern  issues. 

Post-modernism  holds  promise  for  guiding  research  and  development  in  instructional  technology.  The 
diversity  of  theories  and  concepts  in  our  field,  many  of  which  have  been  borrowed  from  other  disciplines,  are  often 
broadly  organized  around  three  distinct  areas  of  inquiry  such  as  behaviorism,  cognitivism  and  constructivism 
(Ertmer  & Newby,  1993).  Accordingly,  the  theoretical  perspectives  in  our  field  do  not  always  enrich  each  other 
because  research  is  often  aligned  with  a particular  orientation.  While  there  is  recognition  that  no  single  theoretical 
perspective  can  completely  explain  learning  (Snelbecker,  1989;  Smith  & Ragan,  1993),  the  potential  relationships 
between  differing  views  could  lead  to  stronger  theories,  a promise  that  could  be  realized  through  an  affirmative 
orientation  to  post-modernism. 

Finn  (1962/1996)  believed  that  technology  was  “...  a way  of  thinking  about  certain  classes  of  problems  and 
their  solutions”  (p.  48).  Post-modernism  offers  a way  to  think  about  thinking.  It  may  not  be  everyone’s  “cup  of 
tea,”  but  our  field  can  be  enriched  by  exploring  alternative  views.  At  the  very  least,  post-modernism  challenges  us 
to  understand  our  own  beliefs  and  recognize  how  our  values  are  represented  in  everything  that  we  do. 
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Abstract 

The  purpose  of  this  study  is  to  compare  direct  instruction  with  anchored  instruction  based  on  students ' 
learning  performances  and  attitudes  (n~100)  in  teaching  basic  geographical  concepts  in  an  introductory 
undergraduate  level  course.  Direct  instruction  students  will  attend  their  regular  lecture-based  class  sessions. 
Anchored  Instruction  students  will  design  a lesson  to  teach  the  basic  geographical  concepts.  It  is  expected  that 
students  in  the  both  groups  will  perform  equally  on  the  performance  posttest  and  the  students  in  the  anchored 
instruction  group  will  score  better  than  the  ones  in  the  direct  instruction  group  on  the  attitude  questionnaire. 

The  Effect  of  Anchored  vs.  Direct  Instruction  on  Students’  Learning  Basic  Geographical 
Concepts 

In  recent  years,  the  effectiveness  of  direct  instruction  and  anchored  instruction  has  been  a hot  debate 
subject  among  researchers.  When  comparing  with  other  instructional  settings  the  researchers  supporting  the 
constructivist  notion  advocate  that  learning  significantly  occurs  and  is  efficiently  transferred  through  anchored 
instruction.  Students  also  maintain  more  positive  attitudes  towards  the  instruction  (CTGV,  1992).  On  the  contrary 
some  research  reveals  similar  findings  for  direct  instruction  (Shuman,  1999;  Bergin  & Walworth,  1999) 

Anchored  instruction  is  a teaching  paradigm  which  advocates  that  some  subjects  can  be  taught  in  the 
context  of  a situation  in  which  that  subject  is  applicable.  The  purpose  of  employing  an  anchored  instruction  model  is 
to  create  interesting,  realistic  contexts  through  technology  or  other  means  that  encourage  the  active  construction  of 
knowledge  by  learners  instead  of  learning  just  facts  and  principles  (Bransford  et  all.,  1990;  Fen  at  all.,  1997). 

Anchored  instruction  is  based  on  situated  learning  and  focuses  on  problem  solving.  Situated  learning  is  a 
general  theory  of  knowledge  acquisition,  promoting  the  idea  that  knowledge  needs  to  be  presented  in  an  authentic 
context  and  that  learning  requires  social  interaction  and  collaboration.  Authentic  contexts  provide  an  opportunity  for 
students  to  experience  the  problems  or  immerse  in  the  phenomena  being  investigated  to  produce  knowledge  and 
solutions  similar  to  the  ones  experts  do.  Moreover,  the  learners  within  authentic  environments  are  able  to  develop 
mental  models  of  their  authentic  experiences  for  positive  learning  transfer  (CTGV,  1990,  1992). 

Direct  instruction,  a behaviorist  or  instructivist  approach  in  teaching  and  learning,  is  based  on  the  notion 
that  learning  can  be  facilitated  through  clear  instructional  presentations  which  rule  out  likely  misinterpretations  and 
facilitate  generalizations.  Also,  direct  instruction  focuses  on  teaching  and  practicing  sub  skills  and  sub-knowledge 
that  allow  students  to  advance  to  higher-order  skills  (Kozloff  et  all,  1999;  Direct  Instruction,  1999).  As  a teaching 
strategy,  direct  instruction,  on  the  other  hand,  is  a systematic  and  highly  structured  instructional  process  that  leads 
students  to  master  in  an  extremely  efficient  manner  (DiGhiera,  1998).  Some  key  components  of  this  process  are 
scripted  lesson  plan  that  is  evaluated  and  revised,  curriculum  designed  to  build  new  skills  on  previously  learned  ones 
and  small-group  sessions  where  teacher  and  students  interact.  (Direct  Instruction,  1999). 

Geography  education  lacks  of  empirical  research  that  might  provide  guidelines  for  practitioners  to  evaluate 
alternative  teaching  and  learning  strategies  (Downs,  1994).  Very  few  studies  investigated  the  effectiveness  of 
different  instructional  techniques  in  terms  of  students*  attitudes  and  learning  performance.  Nordstrom  (1996) 
compared  three  instructional  approaches,  a traditional  lecture-based  direct  instructional  approach,  a hybrid  half- 
lecture half-collaborative  approach  and  a fully  collaborative  approach,  to  teach  an  undergraduate  environmental 
management  course.  Based  on  the  students’  evaluations  of  the  course,  he  found  that  “the  students  view  collaborative 
approach  as  a viable  alternative  to  more  traditional  approach”.  On  the  other  hand,  another  study  comparing 
traditional  lecture  approach  with  cooperative  learning  to  teach  an  introductory  geography  class  by  Hertzog  and 
Lieble  (1996)  revealed  the  result  that  there  was  no  significant  difference  between  the  two  approaches  based  on  the 
students’  posttest  performance  results.  Additionally,  students  in  the  lecture  group  and  cooperative  group  exhibited 
similar  attitudes  towards  both  teaching  techniques  (the  mean  attitude  scores  were  2.9  and  2.8,  consecutively) 

The  purpose  of  this  study  is  to  compare  direct  instruction  with  anchored  instruction  based  on  the  students’ 
learning  performances  and  attitudes  in  teaching  basic  geographical  concepts  in  an  introductory  undergraduate  level 
course  to  provide  practitioners  with  empirical  data  on  alternative  geography  teaching  and  learning  strategies.  Based 
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on  the  literature  review  above,  it  is  expected  that  the  students  in  both  groups  will  perform  equally  on  the  posttest. 
The  students  in  the  anchored  instruction  group  will  score  better  than  the  ones  in  the  direct  instruction  group  on  the 
attitude  questionnaire 

Method 

Subjects 

The  subjects  in  this  study  are  100  undergraduate  freshmen  and  sophomore  students  enrolled  in  four  sections 
of  an  introductory  geography  course  at  a university  in  the  northeast  Florida.  Each  section  has  25  students  and  the 
gender  breakdown  is  60%  male  and  40%  female.  Two  of  the  sections  are  randomly  assigned  to  the  direct  instruction 
group  and  the  other  two  groups  are  assigned  to  the  anchored  instruction  group.  The  participation  in  the  experiment  is 
considered  a course  requirement,  but  the  assessment  results  are  not  considered  a part  of  the  course  grades. 
Instructional  Materials 

There  are  no  specific  instructional  materials  to  be  used  during  the  study.  For  the  direct  instruction 
intervention,  the  direct  instruction  group  will  attend  their  regular  lecture-based  class  sessions,  and  the  instructors 
will  use  their  regular  class  notes  and  handouts  to  teach  the  basic  geographical  concepts,  such  as  the  definition  of 
geography,  places  and  regions,  physical  systems,  human  systems,  environment  and  society,  etc.  For  the  anchored 
instruction  intervention,  the  students  in  the  anchored  instruction  group  will  be  given  a scenario  in  which  they  will  be 
asked  to  design  a lesson  based  on  a template  to  teach  the  basic  geographical  concepts.  The  template  is  the  one  taught 
in  the  introductory  educational  technology  course  at  the  university.  The  lesson  plan  template  requires  students  to 
consider  and  specify  instructional  goals  and  objectives,  instructional  materials  to  be  used  during  the  instruction, 
learner  level  and  learner  characteristics,  instructional  procedure  including  motivating  students,  helping  students 
recall  prerequisites,  presenting  information,  providing  practice  and  feedback,  and  student  assessment  with  the  actual 
assessment  tool.  The  students  will  be  free  to  use  all  the  resources  to  design  the  lesson,  including  the  Internet  and  the 
class  textbook  as  well  as  other  textbooks. 

Independent  Variables 

The  independent  variables  in  the  study  are  the  types  of  instructional  procedures:  Direct  instruction  and 
anchored  instruction. 

Direct  instruction. 

Direct  instruction  will  be  used  to  provide  students  with  direct  information  through  a lecture  to  teach  the 
basic  geographical  concepts. 

Anchored  instruction. 

Anchored  instruction  is  designed  to  provide  a contextualized  learning  environment  where  the  students  are 
put  in  an  instructor  role  and  required  to  design  a lesson  plan  on  basic  geographical  concepts  to  teach  in  an  actual 
classroom  setting. 

Dependent  measures 

There  are  two  types  of  dependent  measures  in  this  study:  Students’  learning  performance  scores  and 
students’  attitude  questionnaire  scores  at  the  end  of  each  instruction.  Students’  learning  performance  will  be 
measured  by  a multiple-choice  performance  test  that  has  50  items.  Students’  attitudes  towards  the  instructional 
procedures  will  be  measured  by  Keller’s  34-item  Course  Interest  Survey  (Keller,  1995)  based  on  students’  attention, 
relevance,  confidence  and  satisfaction.  Both  performance  test  and  Course  Interest  Survey  have  high  reliability  and 
validity. 
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Abstract 

This  paper  describes  the  background  to  the  problem  and  methodology  of  a study  that  explored  a reflective 
new  media  pedagogy  through  the  design  and  production  of  a web-based  learning  environment.  This  environment 
addresses  and  invites  cancer  patients  and  their  supporters  to  question  established  social  and  cultural  discourses 
that  shape  the  experiences  of  cancer  by  inviting  visitors  to  create  ways  to  talk  about  cancer  that  more  closely  relate 
to  the  uniqueness  of  their  lives  and  to  the  characteristics  of  the  disease.  Through  a collaborative  approach  to 
educational  media  design  involving  an  educational  media  designer  and  cancer  patients  and  their  supporters,  a web 
site  (http://www.cancershock.com)  was  produced  to  support  cancer  patients  and  their  supporters  in  creating  non- 
linear, flexible  discursive  spaces  for  rewriting  social  and  cultural  discourses  that  shape  and  inform  the  experience 
of  cancer  in  Western,  industrialized  cultures.  This  interdisciplinary  study  presents  a design  practice  influenced  by 
cultural  studies  approaches  to  new  media  pedagogy  and  drawn  from  the  experience  of  engaging  in  collaborative 
and  reflective  educational  new  media  design. 

Introduction 

Seriously  ill  people  are  wounded  not  just  in  body  but  in  voice.  (Frank,  1995,  p.  xii) 

If  I were  to  demystify  or  deconstruct  my  cancer,  I might  find  that  there  is  no  absolute  diagnosis, 

no  single  agreed-upon  text,  but  only  the  interpretation  each  doctor  and  each  patient  makes. 

(Broyard,  1992,  p.  21) 

The  most  disorienting  challenge  to  traditional  thinking  posed  by  developments  in  the  postmodern 

era  is  the  perception  that  illness  is  no  longer  a purely  biological  state  - no  longer  a brute  force  of 

nature  - but  rather  something  in  part  created  or  interpenetrated  by  culture.  (Morris,  1998,  p.  70) 

This  paper  describes  the  background  to  the  problem  and  methodology  of  a study  that  explored  a reflective 
new  media  pedagogy  through  the  design  and  production  of  a web-based  learning  environment.  This  interdisciplinary 
study  presents  a design  practice  influenced  by  cultural  studies  approaches  to  new  media  pedagogy  and  drawn  from 
the  experience  of  engaging  in  collaborative  and  reflective  educational  new  media  design.  It  uses  the  design, 
production,  and  revision  of  a web  site,  http://www.cancershock.com  (hereafter  referred  to  as  “CancerShock”),  as  an 
educational  intervention  to  study  the  educational  possibilities  presented  by  a culturally  informed  new  media 
pedagogy.  CancerShock  acts  on  behalf  of  cancer  patients  and  their  supporters  to  reffame  notions  of  health,  health 
care,  and  disease  in  ways  that,  compared  to  choices  presented  by  traditional  Western  healthcare  practices,  are  more 
supportive  of  the  uniqueness  of  the  daily  lives  of  cancer  patients  and  their  supporters. 

The  pedagogy  of  CancerShock  parallels  a growing  body  of  thought  that  calls  for  rethinking  the  care  of 
chronic  diseases  like  cancer  in  ways  that  have  stronger  connections  to  the  cultural,  subjective  uniqueness  of  a 
patient’s  lived  experiences  (Broyard,  1992;  Frank,  1995;  Kleinman,  1988;  Radley,  1993;  Stacey,  1997;  Treichler, 
1992).  This  literature  often  refers  to  the  notion  of  the  “postmodern”,  (Morris,  1998),  “postcolonial”  (Frank,  1995)  or 
“participatory”  (Stacey,  1997)  patient  as  one  who  actively  creates  their  own  ways  (through  non-linear  personal 
narratives)  of  living  with  the  disease  versus  being  positioned  by  the  pre-dominant  rational,  linear  discourses  of 
biomedicine.  This  study  presents  a culturally  informed  collaborative  approach  to  educational  technology  design  that 
tries  to  support  a cultural  intervention  in  healthcare  and  support  discourses  while  offering  a critique  of  current 
educational  technology  design  models. 

My  critique  of  dominant  forms  of  educational  media  design  influenced  by  educational  psychology  shows 
an  alternative  approach  that  is  guided  more  by  theoretical  frameworks  contained  within  cultural  studies  (Grossberg, 
Nelson  & Treichler,  1992)  approaches  to  media  study  including  poststructuralism  and  feminisms.  The  methodology 
of  this  study  structures  its  inquiry  around  discursive  contextualizations  of  the  social  and  cultural  issues  surrounding 
cancer  treatment,  as  well  as,  relating  how  these  discourses  form  and  interact  with  the  participant’s  cultural  models 
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about  healthcare.  I use  those  discursive  contextualizations  to  inform  my  collaborations  with  audience  members  in 
the  design  and  revision  of  CancerShock. 

The  design  principles  that  I developed  for  this  study  serve  as  a response  to  and  a critique  of  traditional 
educational  media  design  methodologies  that  seek  to  create  media  that  deliver  just  the  “right”,  predictable  message 
to  a knowable  and  definable  audience.  In  other  words,  I worked  with  design  principles  that  work  under  the 
assumptions  that  one  can  never  predict  the  ways  that  an  audience  will  interpret  a media  text  and  that  media 
interpretation  occurs  at  the  fluid  borders  of  the  audience's  subjectivities  (Ang,  1996;  Fiske,  1989a;  Fiske,  1989b) 
rather  than  the  center  of  artificially  created,  essentialized  characteristics.  These  design  principles  are  derived  from  a 
reflective  practice  (Schon,  1983)  that  continuously  works  to  resist  facile  descriptions  of  the  audience. 

This  study  is  inspired  by  the  lived,  currently  unfolding,  experiences  of  a friend  who  is  undergoing  treatment 
for  ovarian  cancer.  In  the  summer  of  1998,  a friend  (who  I will  call  M.)  and  media  collaborator  was  diagnosed  with 
ovarian  cancer.  Alienated  and  angered  by  her  encounters  with  dominant  biomedical  discourses  and  wanting  a way  to 
“rewrite”  and  make  sense  of  her  life  with  cancer,  she  became  interested  in  reframing  her  experience  from  one 
dominated  by  the  fragmented  linearity  of  her  health  care  to  one  that  supports  her  fight  against  a disease  that  is 
profoundly  non-linear  and  requires,  not  fragmentation,  but  a constantly  responsive  adjustment  and  juxtapositioning 
of  scenarios,  treatments  and  interpretations.  Her  approach  to  cancer  underscores  a trend  in  people’s  approaches  to 
the  treatment  of  serious  diseases  in  which  patients  are  demanding  a greater  voice  in  the  treatment  process  and 
resisting  the  silencing  and  alienating  practices  of  traditional  medical  discourses  (Frank,  1995;  Kleinman,  1988; 
Radley,  1993;  Stacey,  1997).  The  intervention  of  this  study  is  informed  not  only  by  M.’s  experiences  but  also  by  my 
collaborations  with  other  people  with  cancer  and  their  supporters,  as  well  as  by  emerging  discourses  currently  being 
constructed  by  cancer  patients  about  health  and  disease  (Stacey,  1997).  This  is  especially  the  case  in  light  of  current 
communication  technologies  like  the  Internet  that  allow  easy  access  to  information  and  provide  for  more 
opportunities  to  participate  in  support  communities  (Klemm,  1998).  Because  this  study  involved  collaboration  with 
other  cancer  patients  and  various  family  and  friends  of  cancer  patients,  this  intervention  spoke  not  only  to  M.’s 
situation,  but  to  a wider  audience  of  people  with  cancer  and  their  supports. 

CancerShock  was  created  to  act  as  a model  and  workshop  to  help  people  with  cancer  to  reframe  their 
healthcare  experiences  in  ways  that  resist  the  binaries  that  are  presented  to  people  with  cancer  and  their  supporters. 
These  binaries  include  sick/healthy,  patient/doctor,  feeling  good/feeling  bad  - binaries  that  serve  to  create  static 
notions  of  disease  and  people  with  disease  that  allow  and  disallow  specific  ways  of  experiencing  oneself  with 
cancer.  Through  a collaboration  with  M.  and  a group  of  approximately  thirty  collaborators  living  with  cancer  or 
supporting  someone  with  cancer.  I created  a social  and  cultural  intervention  that  problematizes  narratives  about 
health,  healthcare,  disease,  and  quality  of  life  that  are  presented  to  people  affected  by  cancer  (Frank,  1995; 
Kleinman,  1988;  Morris,  1998)  and  rewrites  those  narratives  so  that  they  better  support  people’s  fight  against  cancer 
and  their  struggle  to  maintain  particular  senses  of  self  and  social  positionings  (Broyard,  1992). 

The  collaborations  were  used  as  a feedback  and  revisions  mechanism  for  the  web  site  through  the  use  of  an 
action-research  oriented  approach  (Stringer,  1999)  to  educational  media  design  that  seeks  to  work  with  members  of 
the  intended  audience  to  create  the  design,  structures,  and  messages  of  the  media.  This  approach  allowed  me  to  be 
reflective  about  my  role  as  an  educational  media  designer  in  ways  that  are  responsive  to  the  multiple  subject- 
positions  of  educational  media  audiences.  The  design  and  production  process  included  ongoing  prototyping  and 
discussions  with  cancer  patients  and  their  supporters. 

Within  the  context  of  this  study,  the  issues  that  were  addressed  revolve  around  how  to  use  media  as  an 
educational  tool  to  trouble  and  disrupt  commonplace  narratives  and  discourses  about  health  and  disease  and  to  create 
new  spaces  in  which  to  rethink  the  limiting  ways  that  people  talk  about  cancer.  The  stories  that  patients  are  told  by 
medical  professionals  and  that  are  implied  by  the  time,  space  and  aesthetics  of  cancer  therapies,  hospital  rooms,  and 
doctors'  offices  are  the  sites  of  intervention  for  CancerShock.  CancerShock  provides  a set  of  activities  and  tools  that 
offer  its  visitors  (i.e.  people  with  cancer  and  their  supporters)  an  opportunity  to  (re)frame  and  (re)tell  stories  about 
fighting  cancer. 

The  questions  of  this  study  related  strongly  to  pedagogy  and  drew  parallels  between  the  art  of  teaching  as  a 
social  and  cultural  approach  and  the  science  of  teaching  as  an  educational  psychology  approach.  I developed  a 
notion  of  new  media  pedagogy  that  is  based  on  the  idea  of  pedagogy  as  an  artistic  practice,  a component  of 
instructional  design  often  alluded  to  by  traditional  cognitive-based  theorists  but  rarely  explored.  Michael  Milano  and 
Diane  Ullius  in  describing  their  book,  Designing  Powerful  Training:  The  Sequential-Iterative  Model , a traditional 
instructional  design  book  write: 
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The  truth  is  that  designing  training  is  part  art,  part  science;  it  requires  both  technique  and 
creativity.  The  very  best  training  designs  exhibit  a kind  of  alchemy,  a transformation  of  content 
into  activities  that  meet  specific  behavioral  objectives.  (Milano  and  Ullius,  1 998) 

In  this  study  I began  to  name  the  artful  parts  of  pedagogy  as  they  relate  to  the  design  of  educational  new  media.  My 
pedagogical  goal  for  this  study  was  to  find  a new  media  pedagogy  that  is  artful  and  therefore  attentive  to  the 
interaction  of  emotions,  memory,  pleasure,  desire,  aesthetics,  representation,  and  learning.  These  are  all  aspects  of 
pedagogy  that  are  considered  in  a cultural  studies  approach  to  pedagogy  (Todd,  1997).  Sharon  Todd  (1997)  captures 
this  notion: 

Moving  away  then,  from  essentialized  views  of  oppression  and  subjecthood,  pedagogy  may  be 
rethought  as  a process  that  gets  tangled  up  in  the  nexus  of  social  relations  where  identifications, 
fantasy  and  desire  begin  to  emerge  as  pressing  concerns  with  the  field  of  the  social,  (p.  4) 

In  fact,  cultural  theorists  of  medicine  have  identified  desire  as  the  sick  person’s  best  medicine  (Broyard, 
1992).  Because  qualities  such  as  desire  are  not  easily  definable,  I explored  a pedagogy  that  resists 
instrumentalization,  an  academic  practice  of  containment.  I developed  a pedagogy  for  this  particular  project  that  acts 
as  one  way  of  approaching  educational  media  design  from  cultural  perspectives. 

From  a cultural  perspective,  pedagogy  can  be  seen  as  invitations  into  a subject-positions.  Therefore  the  new 
media  pedagogy  that  I have  created  in  this  project  is  attentive  to  the  subject-positions  into  which  it  invited  its 
audience.  The  interplay  between  an  audience’s  subject-positions,  experiences  of  their  disease  and  healthcare,  and 
educational  media  represented  important  pedagogical  moments. 

Background  to  the  Problem 

Cancer  introduces  a myriad  of  complexities  into  the  lives  of  those  it  affects.  The  emotional  and  physical 
struggles  presented  by  this  disease  intersect  with  the  multiple  subjectivities  of  the  people  it  touches.  This  calls  for 
approaches  to  educational  media  design  for  this  audience  that  have  not  been  adequately  considered  by  traditional 
approaches  guided  by  cognitive  learning  theories  (Dick  & Carey,  1996;  Smith  & Ragan,  1999). 

This  project  addresses  the  numerous  ways  that  a person's  multiple  subjectivities  impact  efforts  to  create 
social  interventions  with  respect  to  the  complex  issues  raised  by  the  treatment  of  cancer.  For  this  study, 
subjectivities  will  represent  the  social  positions  that  are  offered  by  discourses  centered  around  race,  class,  gender, 
cancer,  biomedicine,  etc.  Subjectivity  presents  a central  theme  in  a cultural  approach  to  new  media  design. 

An  important  question  that  always  must  be  addressed  when  considering  a media  intervention  such  as  this  is, 
“Why  use  media  instead  of  another  pedagogical  or  non-pedagogical  intervention?”.  What  is  gained  and  lost  in  the 
decision  to  construct  meaning  in  this  manner?  This  question  can  be  addressed  specifically  to  this  project.  Part  of  the 
answer  to  this  question  can  found  in  the  trend  that  more  and  more  individuals  are  educating  themselves  about  cancer 
through  the  Internet  (Eysenbach  & Diepgen,  1999). 

In  this  study  I not  only  explored  using  media  like  the  Internet  to  respond  to  socially  and  culturally 
circulated  discourses,  but  I investigated  how  a dispersed  medium  like  the  Internet  broadens  the  field  of  discourses  in 
which  one  becomes  inscribed  through  the  subject-positions  offered.  If  traditional  biomedical  discourses,  with  their 
centralized  and  hierarchical  structures,  try  to  create  fixed  subjectivities,  how  might  the  dispersed  medium  of  the 
Internet  create  the  potential  to  encounter  beliefs  and  truths  which  we  may  never  have  been  encountered?  The 
Internet,  as  a medium  and  source  of  information  and  communication  raises  many  questions  regarding  traditional 
Western,  modernist  understandings  of  certainty,  binaries,  and  progress.  If  the  Postmodern  critique  of  modernist 
notions  of  progress,  objectivity,  and  binary  truths  (i.e.  sick/healthy)  values  the  world  as  a continuum  (Lyotard, 
1984),  the  Internet  as  a postmodern  process  makes  such  modernist  understanding  less  feasible. 

Many  people  who  are  faced  with  cancer  use  the  Internet  to  research  the  disease  and  to  communicate  with 
others  who  confront  cancer  (Eysenbach,  1999).  A search  on  the  Internet  for  cancer-related  sites  revealed  numerous 
web  sites  that  reinscribe  the  rational  discourses  about  cancer  that  this  study  works  to  disrupt.  Despite  the  plethora  of 
information  that  is  available  on  the  Internet  about  cancer,  as  Braddock,  et  al.  (1999)  found,  fewer  than  ten  percent  of 
all  patient  decisions  are  well  informed.  This  could  be  attributed  to  many  factors  including  poor  information, 
however  studies  have  found  that  a great  deal  of  the  information  provided  on  the  Internet  is  accurate  and  up-to-date 
(Kiley,  1998;  McLeod,  1998). 

It  is  important  to  note  that  much  of  the  medical  literature  focuses  on  techniques  to  evaluate  the  accuracy  of 
Internet-based  cancer  information  (Biermann,  1999;  Gagel,  1999)  but  none  addresses  decision  making  using  these 
techniques.  This  indicates  that  while  many  people  are  getting  information  each  other  about  possible  treatments  they 
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are  not  putting  this  information  to  the  most  effective  use.  Patients  and  their  supporters  are  not  taught  what  to  do  with 
all  this  information.  They  are  aware  of  many  of  their  options  but  are  unable  to  make  the  most  informed  medical 
decisions  within  those  options. 

By  placing  CancerShock  within  the  context  of  other  cancer-related  web  sites,  CancerShock  offers  an 
alternative  set  of  perspectives  that  may  begin  to  provide  new  contexts  to  make  sense  of  information  about  cancer 
and  cancer  treatment  that  is  always  in  flux  and  changing.  The  results  of  this  study  show  that  this  context  could  not 
be  discovered/constructed  by  using  traditional  forms  of  educational  media  design.  An  innovative  form  derived 
through  cultural  theory  was  required. 

The  approach  to  educational  media  design  that  I explored  in  this  study  does  not  recognize  educational 
media  design  from  a strictly  scientific  perspective  in  which  meaning  is  seen  as  being  injected  by  media  into 
audiences  (i.e.  students)  while  researchers  examine  the  results  of  this  “treatment.”  Rather,  I explored  how 
educational  media  designers  might  acknowledge  and  work  with  the  ways  that  media  consumption  is  not  only  a 
cognitive  experience,  but  also  an  embodied  and  culturally-situated  experience  that  actively  engages  and  invokes 
memory,  history,  desire,  and  senses  of  time  and  place  (Ang,  1991;  Ang,  1996;  Fiske,1988;  Morley;  1992;  Radway, 
1984).  This  approach  to  educational  media  design  rests  on  the  assumption  that  the  meanings  people  make  as  they 
use  media  are  highly  situated  within  their  daily  lives  and  are  shaped  by  their  generalized  views  about  the  world 
(Morley,  1986).  on  the  Internet  and  communicating  with 

Cultural  Models 

The  illness  narrative  is  a story  the  patient  tells,  and  significant  others  retell,  to  give 
coherence  to  the  distinctive  events  and  long-term  course  of  suffering.  The  plot  lines,  core 
metaphors,  and  rhetorical  devices  that  structure  the  illness  narrative  are  drawn  from  cultural  and 
personal  models  for  arranging  experiences  in  meaningful  ways  and  for  effectively  communicating 
those  meanings.  Over  the  long  course  of  chronic  disorder,  these  model  texts  shape  and  even  create 
experience.  The  personal  narrative  does  merely  reflect  illness  experience,  but  rather  contributes  to 
the  experience  of  symptoms  and  suffering.  (Kleinman,  1988,  p.49) 

While  people  respond  to  media  through  generalized  worldviews  (Hall,  1973;  Morley,  1980;  Morley,  1991) 
or  cultural  models,  important  insights  can  also  be  gained  by  how  the  knowledge  that  people  construct  using  media 
supports,  contradicts,  and  modifies  their  cultural  models.  My  use  of  cultural  models  was  influenced  by  Strauss  & 
Quinn  (1997)  and  Gee  (1996).  Strauss  and  Quinn’s  conception  of  cultural  models  tries  to  understand  opinion 
formation  by  showing  how  individuals  negotiate  attitudes  based  upon  internal  schemas  that  are  influenced  by  social 
and  cultural  contexts.  These  schemas  can  be  identified  in  the  kinds  of  stories  that  people  tell  about  their 
understandings  of  the  world  (i.e.  disease),  therefore  I continually  drew  parallels  between  cultural  models  and 
narratives  - the  later  providing  an  indication  of  the  former. 

Gee  calls  cultural  models,  “pictures  of  simplified  worlds  in  which  prototypical  events  unfold.”  He  notes 
that  people  are  often  unaware  of  their  cultural  models  and  therefore  cultural  models  appear  to  be  “natural”  and 
inevitable  even  though  cultural  models  vary  across  social  groups  and  change  over  time.  This  “naturalness”  can 
potentially  lead  to  individuals  seeing  cancer  and  its  treatment  in  ways  that  harm  cancer  patients  by  preventing  them 
from  considering  every  possible  medical,  emotional,  spiritual,  and  physical  opportunity  to  fight  the  disease  - or  of 
creating  yet  unimagined  ones. 

According  to  Gee,  “They  [cultural  models]  allow  us  to  function  in  the  world  with  ease,  but  at  the  price  of 
stereotypes  and  routinized  thought  and  perception.”  Cultural  models  form  the  basis  of  choices  and  guesses  about 
meaning  within  particular  communities  (i.e.  healthcare,  media  production).  They  always  include  a conception  of 
what  is  acceptable  and  unacceptable  to  do  within  that  cultural  model.  For  example,  when  one  enters  the  world  of 
biomedicine  to  be  treated  for  cancer,  one  begins  to  take  on  the  cultural  model  of  a cancer  “patient.”  This  causes 
people  to  act  and  speak  in  certain  predictable  ways  about  cancer.  As  Gee  notes,  “It's  not  just  what  you  say,  or  even 
just  how  you  say  it,  it's  always  who  you  are  and  what  you're  doing  when  you  say  it.” 

Strauss  & Quinn's  work  indicates  how  people's  cultural  models  interact  with  each  other  and  how  they 
interact  within  various  contexts.  Cultural  models  vary  in  both  content  and  form.  The  process  of  cultural  model 
interaction  can  take  the  form  of  “integration’”  as  people  modify  their  cultural  models  to  make  them  more  consistent 
with  each  other.  They  also  observe  that  people  are  capable  of  possessing  inconsistent  cultural  models  without 
experiencing  a sense  of  dissonance.  For  example,  someone  who  is  traditionally  questioning  and  skeptical  may  place 
more  unquestioned  faith  in  a doctor  or  medical  institution.  They  call  the  process  of  maintaining  two  inconsistent 
models  “compartmentalization.”  According  to  Strauss  and  Quinn,  belief  systems  are  partly  integrated  and  partly 
compartmentalized.  Because  challenging  received  narratives  about  healthcare  existed  as  an  important  goal  of  this 
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study,  how  the  audience  integrates  and  compartmentalize  the  messages  of  the  media  within  their  cultural  models 
presented  a consistent  focus  of  this  study. 

While  I draw  on  the  work  of  Strauss,  Quinn,  and  Gee  to  frame  cultural  models,  I also  work  with 
postmodern  approaches  to  address  how  people  make  sense  of  texts.  Postmodern  perspectives  that  examine  the  way 
audiences  interpret  media  texts  highlight  the  paradoxical  relationship  between  a text  and  the  contradictory  identities, 
positions,  imperatives,  and  experiences  within  the  audience  (Ellsworth,  1997).  This  study  balanced  generalized 
statements  about  individual's  cultural  models  while  acknowledging  that  people  are  not  fixed  by  their  cultural  models 
and  maintain  agency  through  their  multiple  and  intersecting  subjectivities. 

Educational  Media  Design 

The  scientism  of  psychology  is  motivated  by  a fear  that  the  world  cannot  be  mastered,  i.e.  known 
directly  and  certainly,  with  scientific  method.  What  is  manifested  here  is  a desire  for  ‘presence’ 
where  the  world  can  be  know  in  an  unmediated  way.  The  unmediated  and  therefore  certain 
knowledge  is  considered  possible  in  principle  but,  equally,  the  possibility  of  presence  is  thought  of 
as  always  under  threat;  a threat  whose  source  ultimately  lies  in  mediation  in  its  various  forms. 

(Usher  & Edwards,  1996,  p.  56) 

Whereas  empirical  approaches  to  thinking  about  the  audience  of  educational  media  often  ask  questions  that 
fix  the  audience's  subjectivities  and  cultural  models  by  addressing  them  through  essentialized  descriptions  produced 
by  a “learner  analysis,”  culturally  informed  approaches  to  inquiry  frame  media  interpretation  in  terms  of  an 
individual's  multiple  and  fluid  subjectivities.  For  example,  while  a learner  analysis  is  more  likely  to  reveal 
characteristics  about  potential  audience  members  including  their  skill  levels  and  literacies  (i.e.  reading,  math,  etc)  or 
attitudes  about  a particular  concept,  a culturally  informed  approach  is  more  likely  to  problematize  notions  of  “skill” 
and  “literacy,”  articulating  how  they  are  the  result  of  power  structures  based  on  race,  class,  gender,  ethnicity  etc.  A 
culturally  informed  approach  offers  a language  of  speaking  about  the  audience's  attitudes  about  a particular  idea  (i.e. 
healthcare)  that  could  directly  address  how  their  subjectivities  help  to  form  that  attitude. 

This  approach  is  presented  to  counter  theories  of  media  design  based  on  educational  psychology,  an 
approach  which  represents  the  dominant  discourse  of  educational  media  design  and  as  Usher  and  Edwards  (1996) 
indicate,  is  based  on  notions  of  certainty  and  mastery.  Traditionally,  educational  media  design  has  been  influenced 
by  cognitive  theories  (Reigeluth,  1999)  that  in  varying  degrees  isolate  the  mind  from  the  social  contexts  in  which  a 
person  learns  and  constructs  knowledge.  Developments  in  cognitive  psychology  have  conceded  the  importance  of 
the  social  context  of  learning  in  the  development  of  “situated  cognition”  (Brown,  1989).  Situated  cognition  has 
developed  on  the  assumption  that  authentic  activities  (those  rooted  in  the  culture  and  practices  of  a community)  are 
the  most  effective  activities  for  developing  useable,  robust  knowledge.  While  making  strides  towards  a 
consideration  of  social  and  cultural  processes,  situated  cognition  still  valorizes  the  individual  mind  as  sense  making 
instrument  and  does  not  problematize  what  is  a useable,  robust  language.  For  example,  it  doesn’t  ask,  “knowledge 
that  is  usable  by  whom?  to  do  what?  for  whose  benefit?”. 

The  seemingly  neutral  manners  (i.e.  “unbiased”  in  its  treatment  of  culture,  race,  class,  gender,  etc.)  that  the 
“learner”  is  constructed  using  psychological  descriptions  impacts  the  ways  that  the  design  process  of  educational 
media  is  carried  out,  and  ultimately  in  the  nature  of  the  materials  produced  and  the  methods  that  are  used  to  assess 
individuals  after  undergoing  instruction.  For  example,  many  instructional  design  models  engage  in  a “front  end 
analysis”  (Dick  & Carey,  1996)  in  which  one  part  of  it  attempts  to  summarize  the  characteristics  of  the  learner  based 
on  such  items  as  reading/math  levels,  test  scores,  and  the  results  of  questionnaires.  This  front  end  analysis  leads  to 
descriptions  of  a learner  who  is  targeted  with  a set  of  learning  objectives  that  are  to  be  mastered  through  the 
activities  using  the  educational  media  produced.  This  methodology  often  leads  to  essentialized  descriptions  of  the 
learner  and  narrow,  easily  evaluated  learning  outcomes.  This  is  similar  to  the  way  the  medical  model  traditionally 
treats  patients  (i.e.  based  on  fixed,  observable,  and  assessable  notions). 

Despite  its  aspirations  towards  objectivity,  educational  psychology  nevertheless  represents  a subjective 
treatment  of  the  issues  that  define  it  as  a field  of  inquiry.  Its  definition  of  individuals  as  “learners”  is  based  on 
specific  historical  antecedents  that  have  become  reified  over  time  and  institutionalized  through  social  mechanisms. 
The  labels  that  are  applied  to  groups  and  individuals  in  educational  psychology  carry  with  them  inherent  hierarchical 
power  structures  that  occupy  histories.  “Student,”  “learner,”  “learning,”  “teacher,”  “knowledge,”  “motivation,”  and 
“evaluation”  all  posses  situated  expectations  and  constraints,  yet  such  labels  are  rarely  problematized  in  educational 
psychology  as  they  are  operationalized.  This  reification  and  institutionalization  has  been  an  important,  if 
unconscious,  objective  of  educational  psychology's  desire  to  be  considered  a “normal”  science  (Kuhn,  1970). 
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Practices  within  this  discipline  have  produced  power  structures  between  the  learner  and  those  administering 
(and  evaluating)  the  learning  that  values  certain  ways  of  knowing  by  specific  types  of  “learners”  over  others.  These 
epistemologies  influence  traditional  educational  media  design  models  because  they  are  biased  towards  specific 
instrumental  (i.e.  memorization-driven,  skills  based,  problem  solving,  etc)  ways  of  knowing.  Measuring  and 
enhancing  learner  performance  has  a long  history  in  educational  psychology  and  exists  as  one  of  the  founding 
charges  of  the  field  beginning  in  the  early  20th  century  (Danziger,  1990).  Educational  psychology,  through  its 
“scientific”  claims  of  being  the  defacto  authority  in  measuring  learning,  has  as  its  primary  mission  the  measurement 
of  success  and  failure  (Danziger,  1990).  Educational  psychology  proposes  to  measure  learning  and  its  effectiveness 
through  technologies  that  include  “deficit  models”  and  “learner  motivation”. 

Deficit  models  tend  to  have  strong  racial,  classed,  and  gendered  undertones  (McKay  & Wong,  1996),  yet 
they  are  unable  to  adequately  articulate  why  these  “deficits”  are  structured  into  a particular  educational  setting  and 
the  complex  ways  that  they  intersect  within  the  individual.  While  extensive  literature  has  been  written  about 
pedagogical  issues  relating  to  race,  ethnicity,  class,  gender,  and  sexuality,  little  of  this  literature  has  influenced  the 
theoretical  work  done  in  educational  media  design.  This  study  will  work  towards  bridging  this  gap.  This  is  not  to  say 
that  cognitive  approaches  do  not  address  learning  in  social  settings,  however,  by  their  very  nature  they  are  unable  to 
adequately  theorize  educational  issues  related  to  the  inter-relationships  between  race,  ethnicity,  class,  nation  or 
gender  except  to  look  at  these  items  in  terms  of  quantifiable  measures  of  performance  and  motivation. 

Traditionally,  instructional  design  theorists  have  looked  at  “learner  motivation”  from  the  perspective  of 
narrowly  defined  conceptions  of  motivation.  Keller's  ARCS  model  (Keller,  1983)  is  typical  of  such  an  approach  that 
ignores  how  motivation  and  satisfaction  are  culturally  dependent  (i.e.  influenced  by  cultural  models).  This  study 
examined  learning  using  media  as  the  result  of  desire  rather  than  the  result  of  an  abstract  notion  of  motivation.  Such 
an  approach  borrows  from  theorists  who  have  discussed  the  relationship  between  desire,  media  spectatorship 
(Bordwell  & Thompson,  1997;  Fuss,  1992;  Mulvey,  1975),  and  education  (Todd,  1997).  These  ideas  are  represented 
in  film  and  television  theories  of  narrative. 

Film  theory  provides  insights  into  this  approach  by  addressing  how  notions  of  anticipation,  expectation, 
surprise,  desire  and  voyeurism  are  meaning  making  devices  for  media  interpretation  (Mayne,  1993).  Television 
theories  of  narrative,  in  contrast  to  film  theories,  depart  from  traditional  realist  narratives  that  try  to  construct  a self 
contained,  internally  consistent  world.  In  a realist  narrative  everything  makes  sense  within  the  structure  of  the 
narrative,  but,  of  course,  this  does  not  always  occur  in  everyday  life  where  many  narratives  don't  end  in  tidy 
resolutions.  Television,  according  to  Fiske  (1988)  invites  the  reader  into  “producerly”  relations  with  the  text, 
especially  in  relation  to  the  most  common  type  of  television  narrative,  the  serial.  Soap  operas  offer  a good  example 
of  television  narrative  that  start  from  a place  of  disequilibrium  and  never  reach  a point  of  closure.  Even  series,  like 
situation  comedies,  that  reach  a conclusion  at  the  end  of  each  episode,  never  resolve  ongoing  conflicts.  These 
ongoing  conflicts  either  exist  within  character  relationships  or  the  situations  that  they  encounter  within  their  settings. 

Television  theories  of  narrative  offer  fractured  constructs  of  narrative  due  to  television's  serialization  and 
commercial  breaks.  This  fractured  structure  creates  conceptual  spaces  for  greater  variation  in  interpretation  by  the 
reader.  Television  narrative  according  to  Fiske  (1987),  “must  be  able  to  build  into  it  contradictions  that  weaken  its 
closure,  and  fragmentation  that  denies  its  unity.”  These  fragmented  narratives  serve  as  a more  productive  model 
with  which  to  represent  the  narratives  of  everyday  life  with  cancer  and  provide  an  appropriate  model  to  think  about 
the  design  of  a web-based  learning  environment  such  as  CancerShock.  Such  models  of  narrative  can  better  support  a 
design  process  that  allows  for  multiple  interpretations  of  media  based  on  a person’s  multiple  social  positions. 

Mode  of  Address 

The  analytical  concept  of  “mode  of  address”  was  used  throughout  this  study  as  a lens  with  which  to 
examine  the  social  and  cultural  position(s)  media  offer  to  an  audience,  as  well  as  interrogate  a new  media  pedagogy 
for  the  ways  that  educational  media  offer  meanings  and  understandings.  Mode  of  address  was  developed  by 
Althusser  (1971)  as  a way  to  talk  about  how  discourses  (through  address)  hail  people  into  particular  subject- 
positions  - positions  that  have  implications  for  the  way  that  people  can  and  can’t  exist  in  the  world. 

Ellsworth  (1997)  develops  the  idea  of  mode  of  address  to  raise  questions  about  pedagogy  and  to  ask  “who 
does  a piece  of  media  (or  pedagogy)  think  you  are”  in  terms  of  your  subjectivities  (i.e.  race,  class,  gender,  religion, 
ethnicity,  etc).  She  describes  modes  of  address  as  the  relationship  between  social  aspects  of  a text  (i.e.  media  text, 
structure)  and  the  individual  experiences  of  a text  (i.e.  reader’s  interpretation,  emotional  reaction,  etc.).  While  media 
interpretations  are  varied,  some  readings  are  more  likely  than  others,  depending  on  a media's  mode(s)  of  address,  as 
well  as  a person’s  cultural  models.  Ellsworth  develops  the  parallel  between  mode  of  address  of  media  interpretation 
and  the  mode  of  address  of  pedagogy  and  argues  that,  like  media  texts,  all  pedagogies  miss  their  audiences/students 
because  what  one  learns  is  never  exactly  what  is  taught. 
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The  ways  that  media/pedagogy  miss  their  audience  is  an  important  notion  for  Ellsworth  who  reasons  that 
the  difference  between  address  and  interpretation  is  a productive  space  for  teachers.  She  makes  the  claim  that  this 
space  is  a social  space,  an  uncontrollable  space  that  “bears  the  traces  and  unpredictable  workings  of  the 
unconscious”  (Ellsworth,  1997,  p.  38)  and  a space  that  can  be  put  to  use  by  teachers  through  it’s  indeterminacy.  I 
build  on  Ellsworth’s  work  about  address  to  show  how  the  design  of  educational  new  media  can  use  a culturally 
informed  design  approach  that  works  with  this  productive  space  by  acknowledging  the  ways  that  media  audiences 
are  indefinable  and  educational  media  interpretations  are  uncontrollable. 

Decisions  in  media  design  are  based  on  (often  unconscious)  assumptions  about  who  the  user  is  in  terms  of 
her  or  his  race,  class,  ethnicity,  sexuality  and  gender  and  what  cultural  competencies  s/he  possesses  (Kress,  1999). 
Users  must  take  up  the  offered  metaphors  and  structures  and  negotiate  their  own  meanings.  The  reader  (or  in  the 
case  of  healthcare,  the  patient),  however,  is  rarely  fully  aware  of  who  the  text  (or  treatment)  thinks  s/he  is.  Often, 
multiple  modes  of  address  occur  simultaneously,  adding  layers  of  complexity  to  individual  interpretations, 
confounding  the  intended  logic  of  a text.  These  notions  of  media  interpretation  borrow  from  cultural  studies 
approaches  to  media  theory. 

Cultural  Studies 

Cultural  studies  is  a difficult  field  to  define  because  it  is  not  a traditional  academic  discipline  grounded  in  a 
monolithic  theoretical  foundation.  It  borrows  from  literary  history,  sociology,  history,  linguistics,  semiotics, 
feminisms,  philosophy,  anthropology,  and  psychoanalysis  (Grossberg,  Nelson  and  Treichler,  1992).  Cultural  studies 
has  been  adapted  to  the  countries  in  which  it  is  practiced  including  the  US  (Campbell  & Kean,  1 998)  , the  UK 
(Turner,  1 996),  and  Australia  (Frow  & Morris,  1993).  This  is  significant  because  an  important  component  of  cultural 
studies  is  that  it  defines  culture  as  dynamic  and  based  in  local  contexts  and  histories.  In  addition  to  being  developed 
in  cultural  studies  programs,  it’s  theories  and  methods  are  incorporated  into  traditional  disciplines  (and  have 
disrupted  the  assumptions  and  practices  of  entire  academic  fields)  including  communication  studies,  media  studies, 
history,  anthropology,  sociology,  law,  medicine,  political  science  and  education. 

Cultural  studies  can  best  be  described  as  an  approach  to  inquiry  that  focuses  on  the  connection  between 
social  relations  and  meanings  and  the  ways  that  social  divisions  are  made  meaningful.  This  approach  generally  has  a 
political  intention  that  seeks  to  illuminate  the  subordination  of  one  group  under  another.  In  cultural  studies  the 
production  of  knowledge  is  always  seen  as  either  done  in  the  interests  of  those  who  hold  power  or  done  by  those 
who  contest  that  hold.  According  to  this  perspective  culture  is  partially  framed  as  the  subordination  of  non-dominant 
groups  by  the  interests  of  dominant  groups.  In  addition,  culture  is  also  seen  as  the  resistance  to  this  subordination. 
Culture,  in  this  context,  is  therefore  revealed  as  a site  of  social  struggle.  This  struggle  is  exposed  by  cultural  studies 
to  show  how  class,  race,  gender,  and  other  sources  of  inequalities  are  naturalized  and  represented  in  forms  (often 
through  media)  which  break  the  connection  between  these  and  political  and  economic  inequalities. 

Cultural  studies  approaches  to  the  study  of  media  representations  and  media  interpretations  have  developed 
as  a counterpoint  to  traditional  media  reception  research  that  look  at  generalizable  “media  effects”  (Halloran,  1970) 
(i.e.  watching  medical  shows  like  ER  causes  people  to  have  distorted  perceptions  of  healthcare).  Instead,  cultural 
studies  are  more  likely  to  look  at  the  representations  of  a show  like  ER  and  see  how  people  put  those  representations 
to  use  in  their  day-to-day  lives  (Morley,  1992).  Cultural  studies  approaches  to  media  reception  stress  the  importance 
of  viewing  media  in  the  context  of  one’s  day-to-day  life. 

In  order  to  discuss  the  methods  that  I mobilized  to  develop  the  design,  production  and  revision  of 
CancerShock,  I will  now  review  in  greater  detail  the  questions,  methods,  data  sources,  results  and  implications  of 
this  study.  I’ll  address  the  design  of  the  study,  as  well  as  describe  each  phase  of  the  study. 

Study  Design 

The  study  was  divided  into  three  phases:  1)  Contextual izati on  2)  Design  and  Production  3)  Collaboration 
and  Revision.  The  contextual ization  phase  included  a literature  review  that  examined  the  notion  of  subjectivity  in 
relation  to  discourse  and  agency  and  identified  current  discourses  about  disease  and  healthcare.  The  literature  review 
included  a description  of  how  a culturally  informed  educational  media  design  approach  contrasts  to  traditional  forms 
of  instructional  design,  especially  constructivism.  In -addition,  the  contextualization  phase  explored  what  kinds  of 
cancer-related  web  sites  are  on  the  Internet  and  looks  at  the  present  discourses  surrounding  approaches  to  cancer 
treatment  by  patients  and  supporters  through  an  examination  of  discussions  on  Internet  mailing  lists.  The  design  and 
production  phase  included  the  first  iteration  of  CancerShock  as  it  began  the  production  processes.  The  collaboration 
and  revision  phase  included  work  with  individuals  fighting  cancer  and  their  supporters  to  revise  the  site.  This  final 
phase  used  an  action  research  approach  to  constantly  challenge  the  praxis  of  the  underlying  theories  of  this  study. 
Along  with  this  general  account  of  the  study,  I will  now  include  a more  detailed  description  of  each  phase. 
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Contextualization 

The  contextualization  phase  involved  finding  out  what  discourses  are  made  available  to  people  on  the 
Internet  seeking  information  and  support  related  to  cancer,  as  well  as  examining  what  people  talk  about  (i.e.  with 
what  narratives  and  cultural  models  do  they  identify)  when  using  the  Internet  for  communication  about  cancer.  The 
first  question  was  explored  by  undergoing  a search  using  five  popular  search  engines  (e.g. 
http://www.altavista.com/)  and  Internet  portals  (e.g.  http://www.yahoo.com/)  searching  for  the  word  “cancer.”  The 
top  50  sites  that  the  search  produced  were  coded  by  general  categories  and  summarized.  In  addition  about  25  of  the 
sites  (10%)  were  analyzed  for  their  mode(s)  of  address.  The  results  provided  insight  into  what  was  available  to 
people  who  search  for  cancer-related  information  and  support  on  the  Internet. 

To  answer  the  question  about  how  people  use  the  Internet  for  support  related  to  cancer,  transcripts  of  four 
electronic  mailing  lists  were  examined  for  the  themes  and  narratives  of  cancer  addressed  by  people  on  the  lists.  A 
discourse  analysis  provided  insights  into  the  prevailing  ways  that  people  with  cancer  and  their  supporters  approach 
cancer  treatment.  This  information  was  filtered  through  the  literature  of  cultural  critiques  of  Western  medical  model 
discourses.  The  findings  mirrored  many  of  the  perspectives  presented  in  the  cultural  critiques. 

This  first  stage  of  the  study  supported  the  initial  hypothesis  about  the  social  construction  of  cancer  in 
Western  societies.  It  sustained  the  notion  that  individuals  mostly  subscribed  to  the  subject-positions  and  linear, 
rational  approaches  to  the  disease  that  Western  notions  of  biomedicine  offer.  This  stage  also  revealed  that  new 
narratives  were  emerging  that  offered  people  with  cancer  opportunities  to  create  and  (re)write  their  own  narratives 
about  cancer  that  countered  traditional  and  alternative  medical  (i.e.  holistic  medicine)  healthcare  discourses.  This 
information  proved  useful  in  the  design  and  production  of  the  site  by  underscoring  the  importance  of  narrative  in  the 
way  that  people  make  sense  of  cancer. 

Design  and  Production 

The  second  phase  of  the  study  included  an  initial  design  of  the  web  site  based  on  the  findings  of  the 
contextualization  phase,  and  also  included  work  with  a small  group  of  collaborators  including  M.,  her  family  and 
friends.  A great  deal  of  time  during  this  phase  was  spent  learning  the  technicalities  of  administering  and  creating  an 
interactive,  multimedia  web  site.  The  technical  demands  and  tradeoffs  of  creating  a web  site  constrained  how  one 
articulates  a new  media  design  practice.  The  first  iteration  of  CancerShock  included  both  videos  and  animations  to 
support  the  ideas  that  emerged  from  the  initial  inquiry.  This  milestone  gave  way  to  the  final  and  longest  phase  of  the 
study. 

Collaboration  and  Revision 

Collaboration  and  revision  involved  work  with  thirty  individuals  who,  knowing  the  goals  of  the  site,  were 
asked  to  offer  feedback  on  general  and/or  specific  aspects  of  the  site.  The  collaborators  not  only  provided  feedback 
but  also  became  partners  in  the  production  of  the  site.  Because  many  of  the  collaborators  were  found  through 
postings  on  public  cancer-related  mailing  lists,  most  of  the  communication  with  them  occurred  through  email, 
however,  I was  able  to  find  participants  locally  who  allowed  me  to  personally  observe  them  access  and  use 
CancerShock.  Watching  individuals  interact  with  the  site  proved  extremely  helpful  in  improving  the  usability  and 
interface  of  CancerShock.  In  the  revision  and  collaboration  phase  I received  a range  of  feedback  from  technical  to 
conceptual  to  creative. 

Conclusions 

The  key  elements  that  emerged  in  this  study  were  expressed  through  a reflective  and  reflexive  design 
process  that  emerged  in  the  creation  of  CancerShock.  A new  media  pedagogy  that  incorporates  a social  and  cultural 
sensibility  is  one  that  has  the  capacity  to  be  self-reflexive  in  its  awareness  of  its  mode  of  address  and  assumptions 
and  is  reflective  in  its  capacity  to  collaborate  with  its  audience.  This  collaboration  has  the  potential  to  move  towards 
responsive  media-based  pedagogies  that  not  only  are  able  to  address  their  audiences  multiply  but  are  also  able  to 
represent  content  and  curriculum  as  fluid  and  changing.  The  implications  of  such  an  approach  to  educational 
technology  practice  frame  educational  new  media  as  only  one  element  among  many  in  the  fluid  intersections  of 
subjectivity,  power,  representation,  and  pedagogy  that  occur  while  learning.  The  social  and  cultural  approaches  to 
new  media  pedagogy  that  I have  articulated,  through  the  design  and  production  process  of  CancerShock,  suggest  one 
way  to  shift  the  discourses  of  educational  technology  practice  away  from  the  language  of  certainty  and  predictability 
towards  one  that  makes  uncertainty,  contingency,  and  context  productive  by  attending  to  the  complex  social  and 
cultural  intersections  that  occur  while  learning. 
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Abstract 

The  purpose  of  this  study  was  to  assist  an  instructor  in  facilitating  an  online  course  while  the  class  was 
being  offered,  and  to  provide  timely  interventions  for  improving  the  course  during  the  semester.  A formative 
approach  was  used  to  help  deal  with  unforeseen  issues  in  implementing  the  first  online  course  this  instructor  had 
offered.  Through  data  analysis  of  what  worked  well,  what  did  not  work  well,  and  what  improvements  were  needed, 
issues  related  to  both  students  and  instructor's  perception  of  the  class  were  identified  in  the  findings.  From  a 
student  perspective,  these  issues  included:  online  interactions,  assessment,  and  course  participation.  For  the 
instructor,  there  were  concerns  about  new  pedagogy,  technology,  and  workload.  Based  on  these  issues,  instructional 
interventions  were  suggested  and  an  overall  evaluation  was  conducted  through  an  online  survey.  The  approach 
adopted,  findings  identified,  and  recommendations  made  in  this  study  will  have  implications  for  other  instructors 
and  instructional  designers,  especially  those  teaching  online  for  the  first  time. 

Introduction 

The  Web  is  offering  unparalleled  opportunities  for  student  access  to  learning,  information  and 
communication,  and  thus  has  growing  popularity  as  a primary  medium  in  distance  education  (Crossman,  1997). 
Many  colleges  and  universities  are  racing  to  move  their  conventional  courses  online  (Mclsaac,  1998;  Molenda  & 
Sullivan,  2000).  Despite  the  “virtual  land  rush”  to  grab  space  on  the  Internet  and  the  excitement  of  the  technology, 
many  issues  must  be  resolved  before  instruction  can  be  delivered  successfully  via  the  Web.  This  study  addressed 
some  of  the  prominent  issues  encountered  by  an  instructor  offering  a face-to-face  course  online  for  the  first  time. 

There  are  many  challenges  for  faculty  teaching  online  for  the  first  time.  Using  new  pedagogical  strategies 
and  working  with  unfamiliar  technology  can  pose  some  unique  difficulties.  Often,  the  only  feedback  the  instructor 
gets  about  a course  is  summative,  which  merely  allows  improvement  the  next  time  the  course  is  offered.  Instead  of 
relying  on  data  gathered  at  the  end  of  the  course,  the  researchers  in  this  study  took  a more  participative  role  by 
collecting  data  from  multiple  sources  to  inform  interventions  and  to  deal  with  issues  as  they  arose. 

The  purpose  of  the  study  was  to  assist  an  instructor  in  facilitating  an  online  course  and  provide  timely 
interventions  for  improving  the  course.  This  study  investigated  what  worked  well,  what  did  not  work  well,  and  what 
could  be  done  to  improve  or  solve  the  problems.  The  results  of  this  study  describe  how  formative  methodology  was 
used  to  address  pedagogical,  technological,  and  communication  concerns  in  the  course.  The  findings  are  relevant  to 
instructional  designers  and  instructors  involved  in  web-based  distance  education. 

Web-based  instruction 

One  phenomenon  seen  in  the  realm  of  Distance  Education  (DE)  is  the  increased  use  of  the  Web  as  a 
primary  delivery  system  (Crossman,  1997).  When  the  Web  was  first  used  as  a delivery  system,  DE  was  often 
considered  to  be  traditional  correspondence  education.  Instruction  was  (and  in  some  cases  still  is)  delivered  through 
pages  full  of  self-instructional  text.  Interactivity  was  quite  limited  (Hirumi  & Bermudez,  1996). 

A number  of  case  studies  have  provided  some  guidelines  and  heuristics  on  building  WBI.  But  an  extensive 
literature  research  of  DE  survey  models  in  early  1998  revealed  that  there  were  no  validated  models  specific  to  DE 
(Williams,  Paprock,  & Covington,  1999).  Fitting  distance  learners'  needs  still  presents  a challenge  for  distance 
educators  given  the  differences  of  needs,  ages,  cultural  backgrounds,  interests,  and  educational  levels  (Willis  & 
Dickinson,  1997).  Distance  educators  and  researchers  are  exploring  more  theoretical  frameworks  to  guide  the 
planning,  design,  development,  and  delivery  of  Web-based  instruction. 


442 


449 


Pedagogical  issues  that  arise  in  WBI  include  information  overload,  “lost  in  hyperspace,”  feelings  of  a lack 
of  cohesiveness,  and  technical  problems.  Among  institutional  issues,  the  amount  of  time  for  an  instructor  to  prepare 
and  deliver  the  course  seems  to  be  a one  of  the  biggest  concerns  (Hill,  1997).  While  some  guidelines  have  been 
established  on  how  to  facilitate  online  learning  (Eastmond,  1995;  Harasim,  1993),  there  is  a scarcity  of  research 
studies  on  implementation  and  how  formative  evaluation  can  help  solve  unforeseen  problems. 

Formative  Evaluation 

Formative  evaluation  (FE)  is  a process  of  collecting  empirical  data  during  the  developmental  stage  for 
revision  and  improvement  of  instructional  products  (Guba  & Lincoln,  1985;  Weston,  McAlpine,  & Bordonaro, 

1995) .  The  primary  goal  of  FE  is  to  improve  the  quality  of  the  product  being  developed  so  that  the  desired  goals  of 
the  product  will  be  met.  The  evaluation  process  involves  assessment  as  well  as  judgment  based  on  the  specific 
information  collected  from  the  users  (Beyer,  1995).  Conventionally,  FE  has  been  conducted  before  implementing 
the  instructional  resources  or  curricula.  The  authors  of  this  study,  however,  advocate  using  FE  as  an  ongoing  process 
in  the  implementation  stage  for  distance  education. 

Methodology 

Research  framework 

Given  the  inexperience  of  the  instructor  in  online  teaching  and  the  novelty  of  the  course  delivery  system, 
improvements  were  needed  throughout  the  course.  To  ensure  the  effectiveness  of  the  class,  timely  interventions 
were  crucial.  Due  to  the  nature  and  purpose  of  the  study,  action  research  was  adopted  as  the  framework.  The  action 
research  framework  was  used  to  guide  the  researchers  to  understand  the  problem,  develop  strategies  for  solving  the 
problems,  and  transform  them  into  best  practices.  The  research  study  involved  three  cycles  of  discovery, 
intervention,  and  evaluation  (Byrant  1996).  It  was  a constant  process  of  observation,  reflection,  and  action  (Stringer, 

1996) .  Action  research  methodology,  when  used  in  the  context  of  Instructional  Technology,  involves  an  ongoing 
process  of  formative  evaluation.  This  approach  was  used  to  gather  functional  feedback  by  learning  from  the  event  as 
they  occurred  and  promoted  positive  changes  in  a timely  manner. 

Research  context 

This  research  study  was  carried  out  in  an  undergraduate  business  course  being  offered  through  a mid-size 
Midwestern  university.  The  course  was  offered  via  Web-based  courseware,  Blackboard.  The  courseware  was 
introduced  on  the  campus  in  the  fall  of  1999  and  was  being  used  for  the  first  time  with  this  class.  Fifteen  students 
registered  for  this  class.  The  majority  of  them  were  taking  an  online  class  for  the  first  time.  The  professor  had 
taught  face-to-face  for  over  twenty  years  but  this  was  the  first  time  he  had  taught  the  class  at  a distance.  An 
instructional  designer  assisted  with  instructional  strategies  and  technical  concerns;  however,  there  were  a number  of 
issues  that  needed  to  be  addressed  by  using  data  gathered  during  the  course. 

The  instructor  used  Problem-Based  Learning  to  facilitate  student-centered,  self-directed  learning.  His  plan 
was  to  prompt  students  with  real  world  questions  and  have  groups  lead  class  discussions  for  the  particular  topics 
being  addressed.  Class  activities  included  individual  discussions  and  small  group  assignments.  Individual  activities 
involved  participating  in  online  discussions  by  answering  questions  proposed  by  the  instructor.  Group  activities 
consisted  of  primarily  mini-cases  appropriate  to  the  chapter  in  the  textbook  for  that  particular  week.  The  class  was 
randomly  divided  into  three  groups  with  five  students  in  each  group.  Students  were  graded  by  their  performance  on 
two  online  exams  (with  multiple  choice  questions  and  essay  questions),  individual  online  participation,  and  group 
project  participation. 

Participants 

The  participants  for  this  study  included  all  15  students  enrolled  in  the  course,  the  instructor,  and  the 
instructional  designer,  who  was  responsible  for  instructional  and  technological  support. 

Research  questions 

The  focus  of  this  study  was  to  formatively  evaluate  an  undergraduate  business  course  and  help  the 
instructor  improve  the  course  when  it  was  being  offered.  The  research  questions  guiding  this  study  included: 

• What  issues  do  instructor  and  students  encounter  in  teaching  and  taking  an  online  course  for  the  first  time? 

• How  can  these  issues  be  resolved  in  a timely  manner  to  improve  a course  as  it  is  being  offered? 

• What  recommendations  can  be  offered  for  instructors  teaching  an  online  course  for  the  first  time? 
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Data  collection 

Observations 

Students’  postings  were  observed  to  determine  the  general  trend  of  students’  participation.  Postings 
included  both  individual  replies  to  the  class  discussion  questions  posted  by  the  instructor  and  also  their  discussions 
as  a group  member  while  collaborating  with  others  to  complete  the  group  project. 

Survey 

Two  surveys  were  designed  to  gather  students’  feedback  about  the  class.  The  first  survey  was  administered 
before  the  middle  of  the  semester  via  email.  The  survey  included  three  general  open-ended  questions  in  the  email 
survey:  what  went  well,  what  problems  they  encountered,  and  what  suggestions  they  had  for  improvement.  Nine 
students  out  of  1 5 (60%)  responded. 

The  second  survey  was  developed  based  on  the  responses  from  the  first  survey.  The  interventions 
suggested  by  the  researchers,  and  the  types  of  interaction  in  distance  education  include  the  following:  learner- 
instructor,  learner-learner,  learner-content,  and  learner-technology  (Moore,  1989;  Hillman  et  al.  1994).  This 
anonymous,  Web-based  survey  was  delivered  by  using  the  Assessment  Tool  in  the  course  site.  The  survey  was 
consisted  of  19  multiple-choice  questions  and  three  open-ended  questions.  Twelve  out  of  15  students  (80%) 
responded  to  the  survey. 

Interviews 

The  researchers  carried  out  one  semi -structured  interview  at  the  beginning  of  the  project  and  two  informal 
phone  conversations  with  the  instructor  during  the  semester.  The  first  interview  was  to  obtain  information  about  how 
the  instructor  would  like  to  teach  this  class  and  what  issues  he  was  dealing  with.  Two  informal  phone  interviews 
between  the  middle  and  the  end  of  semester  were  to  solicit  instructor’s  opinions  on  the  collection  of  feedback  from 
students  and  the  proposed  interventions  based  on  the  feedback. 

Document  study 

Email  messages  sent  back  and  forth  between  the  instructor  and  the  students  provided  another  important 
source  for  understanding  the  issues  surrounding  this  class.  Eighty  messages  archived  by  the  instructor  were 
analyzed. 

Data  analysis 

The  data  collected  from  the  observations,  surveys,  interviews,  and  email  messages  were  triangulated  to 
determine  the  students’  and  instructor’s  perceptions  of  the  class,  issues  raised  throughout  the  semester,  and 
recommendations  for  future  improvement.  Numerical  data  were  tabulated  by  survey  items  and  corresponding 
percentage  in  each  item.  The  data  analysis  on  qualitative  data  involved  an  iterative  inductive  process.  The  initial 
data  analysis  started  with  reading  and  rereading  email  messages  and  coding  the  small  complete  text  units  into 
indexed  categories.  Through  iterative  induction  and  constant  comparison  with  categories  and  data,  key  issues  and 
themes  were  generated. 

Findings 

The  findings  presented  in  this  report  reflect  the  process  of  identifying  areas  needed  for  improvement, 
recommendations  for  instructional  interventions,  and  evaluating  the  course  after  the  interventions.  Observations  and 
faculty  interviews  revealed  issues  that  arose  from  delivering  this  first  online  course.  The  email  survey  data,  web- 
based  survey,  and  their  email  communications  with  the  instructor  revealed  the  students’  perception  of  the  class  and 
the  problems  they  encountered.  The  findings  are  divided  into  three  sections  based  on  the  process  of  formative 
approach. 

Early  to  Mid-Semester  - Before  Intervention 

Observation 

After  nearly  three  weeks,  there  had  been  very  few  postings  to  the  course  message  boards.  There  were  only 
48  posts  in  the  first  three  weeks  of  the  class,  an  average  of  less  than  one  post  per  student,  per  week.  Several  of  these 
posts  were  related  to  technical  issues  such  as  where  to  post  the  answers  to  the  individual  questions  and  group 
questions. 

Besides  the  limited  postings,  students  were  answering  the  case  problems  by  copying  out  of  the  book.  Once 
one  person  had  posted  an  answer  to  the  problem  statement,  the  responses  from  each  of  the  team  members  tended  to 
be  very  similar  and  did  not  add  new  information  to  the  discussion.  There  was  not  much  interaction  occurring  among 
the  group  members  except  when  they  were  ready  to  post  their  answers  to  the  class  discussion  board.  In  addition,  one 
of  the  postings  indicated  that  there  was  not  much  communication  occurring  in  other  forms  during  the  early  part  of 
the  semester. 
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After  observing  the  first  three  weeks  of  the  class,  the  researchers  requested  an  interview  with  the  instructor 
to  determine  his  perception  of  the  class  and  to  investigate  whether  other  channels  of  communication  were  being 
used. 

Faculty  Interview 

The  initial  interview  was  conducted  four  weeks  into  the  semester  and  lasted  two  hours.  After  having  taught 
classes  in  the  field  of  business  and  insurance  for  over  twenty  years  in  a face-to-face  format,  the  professor  found  that 
teaching  via  the  Web  presented  a number  of  challenges.  Underlying  many  of  the  specific  issues  that  the  professor 
was  dealing  with  was  a lack  of  time  to  develop  and  work  on  the  class  because  it  was  being  taught  in  addition  to  his 
normal  class  load. 

New  Pedagogy 

When  first  considering  how  to  re-structure  the  class  to  take  advantage  of  the  Web-based  environment,  the 
professor  decided  that  he  needed  a new  pedagogical  approach.  He  researched  a number  of  instructional  strategies 
and  decided  that  a problem-based  learning  approach  would  be  most  appropriate  for  the  content  of  the  course. 
Implementing  this  new  approach,  however,  proved  to  be  more  difficult  than  anticipated. 

The  textbook  traditionally  used  for  the  course  was  the  primary  resource  for  course  material.  The  instructor 
adapted  the  textbook  problems  to  the  online  environment  and  formed  teams  to  work  on  solving  a problem  for  each 
unit.  Using  this  format,  the  professor  hoped  to  stimulate  conversation.  By  doing  so,  he  believed  that  the  students 
would  become  more  engaged  with  the  material  and  would  have  a stronger  learning  experience. 

At  the  time  of  the  first  interview,  there  were  few  indications  that  the  students  were  working  beyond  the 
material  in  the  textbook  to  solve  the  problems.  The  professor  expressed  some  frustration  with  the  lack  of  depth  to 
the  responses  and  did  not  feel  confident  that  this  new  approach  was  as  effective  as  he  had  hoped.  He  did  note, 
however,  that  there  might  have  been  some  ambiguity  in  the  stated  objectives  and  expectations  that  the  students  had 
been  given.  While  he  had  made  clear  that  he  would  base  a portion  of  the  grade  on  class  and  group  participation,  he 
stated  that  he  needed  to  communicate  his  expectations  better  to  the  class. 

New  Technology 

The  technology  used  for  teaching  the  class  was  also  a factor  in  how  well  interactions  were  occurring. 
While  the  professor  was  extremely  competent  using  many  computer  programs  such  as  spreadsheets,  word  processors 
and  email,  he  was  unfamiliar  with  the  development  tools  for  the  Web.  He  stated  that  he  had  done  some  pilot  testing 
with  the  courseware  in  the  previous  semester.  During  the  semester  break,  however,  the  university  had  upgraded  to  a 
new  version,  changing  the  capabilities  and  functionality  of  the  program.  Even  though  the  courseware  had  better 
capabilities  for  supporting  teams,  there  were  some  unforeseen  consequences. 

Not  having  had  the  opportunity  to  test  the  new  system,  the  professor  did  not  anticipate  the  students’ 
confusion  with  what  message  to  post  under  which  board:  the  Group  Discussion  Board  (for  team  use)  or  the  Class 
Discussion  Board.  He  felt  that  these  difficulties  were  hindering  the  classes’  ability  to  communicate  with  each  other 
and  with  him.  Additionally,  he  found  that  he  did  not  have  the  time  to  check  all  of  the  different  areas  where  students 
might  be  communicating  thus  making  it  difficult  to  track  where  students  might  be  having  problems. 

Student  Interactions 

The  analysis  of  postings  by  students  at  an  early  part  of  the  semester  showed  that  there  was  very  little  use  of 
the  message  boards,  which  were  the  intended  mechanism  for  on-line  discourse.  When  asking  whether  students  were 
using  any  other  mechanism  (email,  live  chat,  telephone)  to  communicate,  the  instructor  stated  that  he  did  not  have 
any  indication  that  there  were  other  methods  being  used. 

The  limited  interaction  was  contrary  to  the  instructor’s  goal  that  team  members  would  work  together  to 
solve  the  problems.  The  instructor  noted  that  students  were  reading  the  material  because  their  answers  on  the 
discussion  boards  reflected  their  reading.  They  were  not,  however,  interacting  with  each  other  in  a way  that 
contributed  to  the  in-depth  learning  that  the  professor  was  trying  to  achieve. 

Email  Survey 

The  open-ended  questions  that  were  emailed  to  students  near  the  middle  of  the  semester  gave  insight  into 
their  perceptions  of  the  course.  While  the  questions  were  open  ended,  the  responses  were  centered  on  a few  key 
issues  about  the  class.  Given  the  professor’s  uncertainties  about  how  well  his  methods  and  the  technology  were 
working,  it  was  surprising  to  find  that  nearly  all  of  the  students  who  responded  to  the  survey  had  a very  favorable 
impression  of  the  class. 
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One  of  the  major  findings  in  this  survey  was  that  students  liked  the  freedom  and  convenience  of  taking  a 
class  over  the  Web.  For  some  students,  it  was  the  only  way  they  could  take  such  a class  because  they  had  work  and 
family  commitments.  Many  thought  the  technology  was  being  used  appropriately  to  meet  their  needs  and 
appreciated  the  use  of  message  boards  over  email  or  other  communication  options.  Additionally,  the  courseware 
was  viewed  as  easy  to  use  and  was  liked  by  most  of  the  students. 

While  most  of  the  respondents  were  satisfied  with  the  overall  structure  and  learning  occurring  in  the  course, 
there  were  a number  of  suggested  improvements.  Many  of  the  changes  revolved  around  the  organization  and 
expectations  for  class  communication.  Students  wanted  to  have  concrete  guidelines  for  project  work.  Several 
mentioned  that  they  were  unsure  about  what  was  expected  from  their  group  in  terms  of  final  deliverables.  There 
was  also  uncertainty  about  what  criteria  were  being  used  to  determine  their  individual  participation  in  the  class  or 
within  their  teams.  The  most  often  mentioned  concern  from  the  students  was  that  the  professor  did  not  seem  to  be 
giving  enough  feedback  in  the  course  Website.  Students  wanted  more  ongoing  commentary  from  the  instructor  so 
that  they  would  know  whether  they  were  meeting  expectations. 

Mid-semester  Intervention 

Based  on  the  major  trends  identified,  the  researchers  proposed  several  strategies  to  the  instructor  to 
improve  the  course.  While  the  strategies  were  developed  as  a result  of  the  problems,  they  were  also  designed  to 
work  within  the  limited  time  budget  the  professor  had  to  make  changes  to  the  course. 

The  first  strategy  researchers  recommended  was  to  use  technology  to  improve  interaction.  Improving 
student  interaction  in  the  course  is  a very  complex  issue  and  is  very  important  in  a problem-based  learning  class  with 
a team-based  approach.  Distance  students  needed  a clear  set  of  guidelines  for  how  to  communicate  online.  The 
recommended  guidelines  included  two  parts.  One  was  to  tell  students  what  technology  to  use  and  when.  The 
second  component  in  the  guidelines  was  to  specify  how  the  technology  was  to  be  used.  The  students  who  were  less 
experienced  in  online  communication  did  not  appear  to  understand  what  they  should  post  in  the  different  message 
areas  or  how  to  use  the  tools  effectively. 

The  second  strategy  we  recommended  was  to  provide  more  feedback.  Given  the  time  pressure  faced  by  the 
instructor,  this  was  a more  challenging  problem.  Addressing  the  issues  related  to  student  communication  might  help 
this  by  encouraging  the  behavior  the  professor  had  hoped  to  occur  (more  student-student  interaction).  If,  for 
example,  students  knew  that  there  were  specified  areas  within  the  course  site  where  they  could  get  feedback,  then 
the  professor  could  centralize  his  responses  and  answers  to  one  student’s  questions  which  might  help  others  in  the 
class.  Such  an  approach  would  prevent  the  instructor  from  having  to  go  into  each  discussion  area  for  each  group  and 
for  the  class  to  address  specific  issues. 

Third,  we  recommended  providing  specific  guidelines  for  the  projects.  While  the  syllabus  had  clearly  stated 
objectives  for  the  outcomes  in  the  class,  there  was  little  explicit  information  about  how  evaluation  would  occur.  In  a 
face-to-face  situation,  issues  like  class  participation  can  be  defined  on  an  ongoing  basis,  depending  on  the  lesson  for 
that  day.  Online,  however,  rules  for  participation  have  to  be  more  clearly  defined.  More  precise  project  guidelines 
would  help  clarify  students’  uncertainty  about  what  was  expected  from  them  in  terms  of  group  work. 

Post  Intervention  Data 

Student  Survey 

The  student  survey  was  designed,  in  part,  based  on  issues  found  in  the  interview  with  the  professor  from 
students’  comments  sent  in  via  email.  Twelve  out  of  the  fifteen  students  (80%)  in  the  class  completed  the  survey. 
The  survey  results  helped  confirm  trends  found  in  the  qualitative  data.  The  survey  was  designed  to  assess  students’ 
learning  experience  with  this  online  course,  their  opinions  about  the  online  test,  and  other  issues  that  concerned 
them.  Demographic  data  showed  that  the  majority  (83%)  of  the  students  had  never  taken  an  online  course  before 
this  class.  Interestingly,  none  of  the  students  reported  the  courseware  to  be  difficult  to  use  (learner-technology 
interaction).  The  results  students’  opinions  on  group  work,  students’  learning,  online  tests,  and  instructor’s  feedback 
will  be  summarized  below  (interactions  between  learners,  instructor,  content). 
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Table  J:  Summary  of  Survey  Findings  (The  number  in  Q indicates  the  number  of  respondents) 


Questions 

Strongly 

Disagree 

Disagree 

No 

Opinion 

Agree 

Strong 

Agree 

Our  group  was  able  to  work  effectively  as  a team. 

8%  (1) 

25%  (3) 

8%  (1) 

42%  (5) 

17%  (2) 

I learned  more  by  doing  this  project  collaboratively  with 
other  team  members  than  I would  have  on  my  own. 

17%  (2) 

50%  (6) 

8%  (1) 

17%  (2) 

8%  (1) 

This  class  met  the  expectations  I had  for  learning  the 
material. 

8%  (1) 

42%  (5) 

17%  (2) 

17%  (2) 

8%  (1) 

I found  online  discussions  to  be  beneficial  to  my  learning 
experience. 

8%  (1) 

33%  (4) 

17%  (2) 

42%  (5) 

0%  (0) 

Online 

Test 

The  content  of  online  test  was  a fair  assessment 
of  what  I was  expected  to  learn. 

8%  (1) 

8%  (1) 

42%  (5) 

42%  (5) 

0%  (0) 

The  format  of  the  test  was  appropriate  for  an 
online  class. 

8%  (1) 

25%  (3) 

33%  (4) 

33%  (4) 

0%  (0) 

The  instructor  provided  sufficient  feedback. 

0%  (0) 

33%  (4) 

25%  (3) 

42%  (5) 

0%  (0) 

The  results  on  group  work  showed  that  59%  (7  out  of  12)  felt  they  were  able  to  work  effectively  with  their 
teams.  When  asked  if  working  collaboratively  with  other  team  members  helped  them  learn  more  than  they  would 
have  working  on  their  own,  67%  (8  out  of  12)  disagreed  or  strongly  disagreed,  one  responded  with  no  opinion  and  3 
agreed.  Related  to  this  finding  was  that  6 students  (out  of  1 1 who  responded  this  question)  did  not  think  that  their 
expectations  for  learning  the  material  had  been  met  through  the  class.  Three  of  them  agreed  the  class  met  their 
expectation,  and  2 responded  with  no  opinion.  Regarding  the  online  discussions,  5 students  (41%)  agreed  that  they 
were  beneficial  to  their  learning  experience;  2 students  (17%)  reported  no  opinion,  and  5 students  (42%)  did  not 
think  online  discussions  were  beneficial  to  their  learning. 

The  class  was  somewhat  divided  over  issues  related  to  an  online  test.  About  16%  (2  students)  of  the  class 
did  not  think  the  test  content  was  a fair  assessment  of  what  they  were  supposed  to  learn,  42%  had  no  opinion  (5 
students),  and  42%  (5  students)  thought  the  test  was  fair.  As  for  appropriateness  of  test  format  for  this  online  course 
(timed  multiple  choice  questions  and  essay  tests),  33%  (4  students)  responded  disagree,  no  opinion,  and  agree 
respectively. 

Students’  opinions  on  instructor’s  feedback  were  also  divided.  Five  students  indicted  that  the  instructor 
provided  sufficient  feedback,  3 students  responded  no  opinion,  and  4 students  did  not  think  the  feedback  was 
sufficient. 

Students’  responses  to  the  opened-ended  questions  revealed  more  detailed  information  about  their 
perception  of  the  course.  These  and  the  analysis  results  from  students  and  instructors’  email  messages  will  be 
summarized  below.  These  qualitative  data  gathered  from  students’  survey  informed  the  quantitative  results  of 
students’  perception  of  the  class. 

Students’  Perspective  on  the  Class 

Several  students  mentioned  that  they  enjoyed  the  flexibility  and  convenience  of  taking  an  online  course. 
They  also  liked  the  professor’s  patience  and  understanding  in  dealing  with  problems  they  ran  into.  Some  students 
enjoyed  the  group  work  and  online  discussions.  Three  prominent  themes  regarding  to  problems  and  concerns 
emerged  from  the  qualitative  data  analysis. 

Technical  concerns 

Although  the  courseware  was  relatively  user-friendly,  students  constantly  ran  into  technical  problems 
throughout  the  semester.  Several  students  had  difficulties  in  taking  the  online  exam.  The  restriction  of  the  exam 
format  and  lack  of  flexibility  in  test  sequence  caused  some  students  not  to  perform  as  well  as  they  expected.  Some 
students  got  a zero  on  particular  sections  due  to  technical  restrictions.  The  instructor  had  to  reset  the  test  for  these 
students.  Other  technical  problems  included  lost  connections  during  the  test,  transferring  files,  and  posting  and 
locating  messages  at  the  right  place. 

Students  reported  in  the  survey  that  they  did  not  think  Blackboard  was  difficult  to  use.  The  email  messages 
sent  back  and  forth  between  students  and  instructor  revealed  that  there  were  more  technical  problems  than  questions 
about  assignments  or  reading.  The  instructor  had  to  spend  a great  deal  of  time  resolving  these  technical  problems. 
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Assessment 

Students  were  very  concerned  about  their  grade,  and  how  they  were  evaluated  on  the  individual 
participation  and  group  participation.  Several  students  emailed  the  instructor  asking  what  grade  they  might  get  and 
how  they  could  improve  their  grade.  In  addition,  students  were  concerned  and  some  were  frustrated  by  the  format 
and  technical  difficulty  of  the  online  test.  More  than  a couple  of  students  mentioned  that  they  were  not  good  test 
takers  even  if  they  knew  the  material  well.  The  timed  exam  made  it  more  difficult. 

Group  work 

This  class  used  small  groups  to  work  on  mini -projects.  It  is  not  surprising  that  there  were  some  complaints 
about  uneven  participation.  “Free  ride”  (no  contribution  to  the  group  task  but  get  the  credit)  was  rather  bothersome 
for  the  students  who  contributed  more  for  the  group  project.  Some  students  were  concerned  about  this  fairness  issue 
in  the  assessment. 

A couple  of  other  issues,  not  reflected  by  majority  of  the  students,  are  worth  mentioning.  Two  students 
thought  that  the  online  class  lacked  social  interaction.  They  felt  they  would  learn  more  through  hands-on  and  face- 
to-face  classes.  In  addition,  insurance  is  a hard  subject  (a  comment  made  by  one  student),  and  that  not  being  able  to 
discuss  it  in  class  made  it  hard  to  understand. 

Instructor’s  Perspective  on  the  Class  , 

Through  our  informal  conversations  and  semi -structured  interview,  the  instructor  expressed  that  formative 
evaluation  was  a very  good  reinforcement.  He  stated  that  it  was  good  to  have  an  independent  source  to  provide 
feedback.  Students  seemed  to  like  this  idea  and  were  quite  open  in  talking  about  their  issues  and  concerns.  He 
thought  a mid-term  formative  evaluation  and  recommendations  were  useful.  If  he  had  time  to  implement  them,  he 
projected  these  recommendations  would  have  helped  improve  this  class.  Based  on  previous  experience,  the 
instructor  had  mixed  feelings  about  his  first  online  course. 

The  instructor  did  not  think  that  students  accomplished  what  he  wanted.  He  expected  them  to  go  beyond 
the  textbook,  but  only  one  or  two  students  did  that  (consistent  with  his  comment  earlier  in  the  semester).  He  also  felt 
that  students  did  not  learn  as  much  as  they  would  have  in  an  on-campus  class.  With  his  on-campus  face-to-face 
class,  he  was  able  to  present  extra  material.  With  this  Internet  class,  although  databases  were  provided,  very  few 
students  used  them.  But  one  of  the  things  that  instructor  really  enjoyed  was  the  rapport  he  and  some  of  his  students 
established  through  emailing  back  and  forth  to  address  questions  and  concerns.  He  found  this  online  rapport  was 
quite  rewarding. 

Discussion 

The  researchers  began  the  study  by  observing  the  visible  interaction  occurring  on  the  course  Web  site. 
Analyzing  the  evaluation  data  identified  three  general  major  difficulties.  First,  very  limited  interaction  occurred  on 
the  course  message  boards  during  early  part  of  the  semester.  One  of  the  reasons  for  this  apparent  lack  of  student 
involvement  was  that  most  of  the  students  had  never  taken  an  online  class  before,  and  they  had  no  experience  in  how 
that  should  communicate  within  the  courseware.  Additionally,  although  the  instructor  was  aware  of  the  lack  of 
student  involvement  in  the  course,  his  limited  time  and  heavy  workload  kept  him  from  putting  more  effort  into 
facilitating  the  students’  online  communications. 

The  second  major  problem  was  the  amount  of  feedback  students  were  getting  from  the  instructor.  The  same 
factor,  time  shortage,  also  affected  the  amount  of  individual  feedback  the  instructor  was  able  to  give  to  the  students. 
The  professor  hoped  that  students  would  interact  more  with  each  other  and  would  learn  from  these  interactions, 
rather  than  relying  on  him  for  the  majority  of  the  feedback. 

The  third  major  issue  that  emerged  was  related  to  standards  for  the  class.  Students  indicated  that  they  were  not  sure 
how  they  were  being  evaluated.  This  uncertainty  was  especially  true  for  the  participation  portion  of  their  grade. 
Clear  and  specific  guidelines  about  how  participation  was  being  evaluated  (for  example,  by  number  of  message 
board  postings,  depth  of  the  posts,  etc.)  would  help  clarify  the  students’  uncertainties.  Likewise,  providing  explicit 
guidelines  for  group  projects  would  have  helped  address  students’  concerns  about  group  work,  which  could  lead  to 
more  participation  from  team  members. 

There  was  a high  degree  of  congruence  between  the  instructor’s  beliefs  and  student  attitudes  on  these  three 
major  issues.  The  instructor,  however,  had  a much  more  negative  view  of  how  the  class  was  progressing  than  the 
students  did.  He  felt  that  the  students  were  going  to  be  very  disappointed  with  the  overall  experience  they  were 
getting  from  the  class.  In  contrast,  email  to  the  research  team  and  other  survey  data  revealed  that  while  students  had 
specific  issues  they  wanted  to  see  improved,  they  were  generally  positive  about  the  class. 
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This  course  presented  a couple  of  challenges  for  the  instructor.  The  first  challenge  was  time  management. 
Due  to  his  busy  schedule  and  heavy  workload,  he  was  not  able  to  facilitate  the  class  discussions  and  group  activities 
as  effectively  as  he  wanted  to.  Much  time  was  spent  on  answering  students’  technical  questions.  If  he  had  been  free 
from  dealing  with  these  technical  responsibilities,  he  would  have  been  able  to  put  more  effort  into  the  development 
of  the  course.  Another  challenge  the  instructor  faced  was  finding  the  best  fit  of  instructional  strategy  to  this 
particular  introductory  course.  There  is  very  little  literature  about  using  PBL  in  Web-based  instruction.  Knowing 
how  to  implement  PBL  in  an  Internet  class  and  how  to  facilitate  group  problem  solving  presented  a great  challenge 
for  the  instructor.  With  limited  guidance  from  the  literature,  one  had  to  explore  the  strategies  through  trial  and  error. 

Conclusion 

The  instructor  in  this  study  was  very  motivated  to  try  innovative  teaching  techniques  and  strategies.  He 
spent  time  reading  pedagogical  literature  and  pilot  testing  the  technology  before  teaching  the  class.  Additionally, 
the  class  was  being  taught  voluntarily,  despite  his  already  busy  schedule. 

Even  with  a highly  motivated  faculty  member,  however,  implementing  an  online  class  is  a difficult  task 
that  requires  new  skills  and  strategies  for  success.  More  importantly,  the  instructor  needs  support  from  the 
university  and  his  or  her  department  to  effectively  work  with  students  at  a distance.  The  support  needed  most  is 
time  to  plan,  build  the  online  courses,  and  ongoing  technical  support. 

As  universities  continue  the  mad  dash  to  offer  their  courses  online,  reasonable  timelines  for  design  and 
development  should  be  taken  into  account.  If  faculty  have  to  use  untested  methods  on  constantly  shifting 
courseware  with  their  students,  sound  instructional  practice  would  suggest  that  this  is  not  good  for  the  students, 
faculty,  or,  in  the  long  run,  the  university.  The  reality  is,  however,  that  there  will  continue  to  be  uncertainty  about 
the  best  practices  for  online  course  delivery  for  the  near  future.  Formatively  evaluating  their  courses  and  making 
improvements  throughout  the  semester  can  reduce  some  of  the  problems  faced  by  faculty. 
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Abstract 

The  purpose  of  this  review  is  to  classify  articles  in  four  Human  Performance  Technology  publications  for 
1997  through  1999.  Two  publications  are  selected  from  each  of  the  two  largest  professional  organizations  for 
performance  practitioners;  the  publishers  are  the  American  Society  for  Training  and  Development,  and  the 
International  Society  for  Performance  Improvement.  Articles  were  classified  according  to  criteria  derived  from 
Klein  (1997),  and  Dick  & Dick  (1989). 

Introduction 

Recently,  a number  of  calls  for  measurement  of  training  effectiveness  have  indicated  that  training  and 
performance  interventions  should  be  expected  to  add  value  in  a way  that  can  be  measured  (Anglin,  et  al,  2000). 
Certainly  the  calls  for  data  are  appropriate;  for  performance  professionals  to  earn  any  credibility,  they  must  lead  the 
way  in  demonstrating  observable,  measurable  outcomes  because  those  outcomes  are  demanded  of  our  customers  for 
their  jobs. 

Calls  for  data  come  from  experienced  practitioners,  distinguished  academics,  and  customers  and  there  is 
now  a greater  focus  and  emphasis  on  development  of  the  profession  as  well  as  related  aspects  such  as  ethics, 
qualifications,  competencies,  and  standards  (Merrill,  et.  al,  1996;  Gery,  1999;  Shrock,  1999;  Kimball,  1999; 
Brethower,  2000;  IBSTPI,  1998). 

The  increased  demands  on  a performance  professional’s  skill  set  and  knowledge  base  require  high  quality 
and  effective  professional  development  opportunities.  Often,  professional  organizations  are  utilized  for  most  if  not 
all  of  an  individual's  professional  development  efforts  as  many  practitioners  read  the  organization's  publications  as 
they  strive  to  improve  their  skills  and  many  attend  an  annual  conference  as  well. 

The  increasing  importance  of  these  publications  was  the  impetus  for  this  review.  A systematic  assessment 
of  the  publications  read  by  numerous  practitioners  is  helpful  for  a number  of  reasons.  As  in  any  performance 
question,  the  current  status  must  be  documented  in  order  to  determine  if  the  goal  is  being  met.  This  review  is  not 
without  precedence;  a number  of  reviews  have  been  conducted  that  examine  the  content  and  scope  of  Instructional 
Technology  publications  (Klein,  1997;  Dick  & Dick,  1989;  Driscoll,  1997;  Driscoll  & Dick,  1999). 

Publication  and  Article  Selection 

The  purpose  of  this  review  was  to  classify  the  articles  in  four  of  the  most  popular  publications  in  the 
Human  Performance  Technology  field,  for  the  period  from  January  1997  through  December  1999.  Those  four 
publications  are:  Training  and  Development,  and  Technical  Training  - both  published  by  ASTD  (American  Society 
for  Training  and  Development),  as  well  as  Performance  Improvement  Quarterly,  and  Performance  Improvement  - 
both  published  by  ISPI  (International  Society  for  Performance  Improvement).  ISPI  and  ASTD  are  currently  the  two 
dominant  professional  organizations  for  practitioners  working  in  the  performance  field. 


Organizational  Missions  (quoted  directly  from  organizational  websites) 

ISPI  - http://www.ispi.org 

“Founded  in  1962,  the  International  Society  for  Performance  Improvement  (ISPI)  is  the  leading 
international  association  dedicated  to  improving  productivity  and  performance  in  the  workplace.  ISPI  represents 


more  than  10,000  international  and  chapter  members  throughout  the  United  States,  Canada,  and  40  other  countries. 
ISPI's  mission  is  to  develop  and  recognize  the  proficiency  of  our  members  and  advocate  the  use  of  Human 
Performance  Technology.  Assembling  an  Annual  Conference  & Expo  and  other  educational  events  like  the 
Institutes,  publishing  books  and  periodicals,  and  supporting  research,  and  recognizing  outstanding  achievements 
through  the  Awards  of  Excellence  program  are  some  of  the  ways  ISPI  works  toward  achieving  this  mission.” 

ASTD  - http://www.astd.org 

“Founded  in  1944,  ASTD  is  the  world’s  premier  professional  association  and  leading  resource  on 
workplace  learning  and  performance  issues.  ASTD  provides  information,  research,  analysis  and  practical 
information  derived  from  its  own  research,  the  knowledge  and  experience  of  its  members,  its  conferences, 
expositions,  seminars,  publications  and  the  coalitions  and  partnerships  it  has  built  through  research  and  policy  work. 
ASTD’s  membership  includes  more  than  70,000  people,  working  in  the  field  of  workplace  performance  in  100 
countries  worldwide.  Its  leadership  and  members  work  in  more  than  1 5,000  multinational  corporations,  small  and 
medium  sized  businesses,  government  agencies,  colleges  and  universities.  ASTD’s  publications  cover  all  aspects  of 
learning  and  performance  in  the  workplace.  Its  books,  magazines,  periodicals  and  reports  address  the  leading 
performance  issues  facing  business  and  the  profession  and  provide  practical  workplace  tools  for  managers, 
professionals  and  technical  workers.” 

Publication  Descriptions 

Performance  Improvement  (PI) 

“PI  is  the  preeminent  journal  focusing  on  today’s  issues  of  individual  and  organizational  improvement. 
Published  10  times  a year,  PI  is  filled  with  information  that  can  improve  your  skills  and  effectiveness.  Since  1963, 
PI  has  consistently  addressed  improving  human  performance  through  a wide  range  of  instructional  and  non- 
instructional  interventions,  as  well  as  imparting  the  tools  of  performance  technology  and  the  challenges  and  new 
frontiers  facing  performance  technologists.  Published  monthly  except  for  combined  May/June  and 
November/December  issues,  this  acclaimed  journal  is  geared  toward  practitioners  of  performance  technology  in  the 
workplace.  Learn  from  hands-on  experiences  with  models,  interventions,  ‘how-to’  guides,  and  ready-to-use  job  aids, 
as  well  as  research  articles.  Performance  Improvement  also  offers  updates  on  trends,  reviews,  and  field  viewpoints. 
The  journal  deals  with  all  types  of  interventions  and  all  phases  of  the  HPT  process.  The  common  theme  is 
performance  improvement  practice  or  technique  that  is  supported  by  research  or  germane  theory.” 

Performance  Improvement  Quarterly  (PIQ) 

“PIQ  is  the  scholarly  publication  of  studies  reflecting  research  in  the  field  of  human  performance  technology.  It 
provides  an  avenue  for  communication  and  stimulates  discussion  among  professionals  in  the  field  who  desire  a 
sophisticated  examination  of  issues  in  human  performance,  instructional  design  and  development,  and  human 
learning.  Now  in  its  11th  year,  this  highly-regarded  publication  continues  to  be  a leader  in  defining  the  field  of 
human  performance  technology.” 

Training  and  Development  (T&D) 

“T&D  is  the  flagship  publication  of  the  industry,”  according  to  the  ASTD  website.  “ Training  & 
Development  is  an  official  publication  of  ASTD,  a not-for-profit  association  of  professionals  in  the  field  of  human 
resource  development,  workplace  learning,  and  performance  improvement.  T&D  is  a monthly  magazine  available  to 
ASTD  national  members  as  part  of  their  dues,  and  also  by  subscription  and  single-copy  sales.  Our  readers  are 
degreed  training  and  development  professionals  and  line  managers.  They  range  from  new  practitioners  to  seasoned 
executives  in  business,  government,  academia,  and  consulting.  Our  goals  are  to: 

• provide  useful,  how-to  information  on  current  best  practices 

• share  new  theories  and  their  applications 

• report  emerging  trends 

• address  relevant  and  pivotal  issues  to  the  field.” 

Technical  Training  Magazine  (TTM) 

“Technical  Training  magazine,  published  by  ASTD  from  1990  to  1999,  is  the  print-based  predecessor  to 
Learning  Circuits.  It  covered  news,  trends,  training  techniques  and  technologies  in  the  technical  training  arena.” 
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Methodology 

The  sample  under  investigation  consisted  of  two  publications  from  each  of  the  two  major  professional 
development  organizations  for  individuals  working  in  training,  ISPI  (Performance  Improvement  Quarterly  and 
Performance  Improvement)  and  ASTD  (Training  & Development  Magazine  and  Technical  Training  Magazine). 
Each  publication  was  reviewed  for  three  calendar  years  — 1997,  1998,  and  1999.  Only  the  informational  articles 
were  reviewed  from  each  issue;  book  reviews,  brief  notes,  editorials  and  commentaries,  and  indexes  and 
bibliographies  were  not  included  in  the  review.  The  publications  contained  800  articles  of  all  types,  700  of  which 
met  the  criteria  of  informational  articles  for  the  review.  The  issues  reviewed  for  each  publication  were: 

• Performance  Improvement  Quarterly,  Volume  10  (1997)  Issues  1-4;  Volume  1 1 (1998)  Issues  1-4;  Volume  12 
(1999)  Issues  1-4; 

• Performance  Improvement,  Volume  36  (1997)  Issues  1-10;  Volume  37  (1998)  Issues  1-9;  Volume  38  (1 999) 
Issues  1-10; 

• Training  and  Development,  Volume  34  (1997)  Issues  1-12;  Volume  35  (1998)  Issues  1-12;  Volume  36  (1999) 
Issues  1-12; 

• Technical  Training  Magazine,  Volume  8 (1997)  Issues  1-8;  Volume  9 (1998)  Issues  1-8;  Volume  10  (1999) 
Issues  1-6. 

Published  articles  were  classified  according  to  specific  criteria  which  were  derived  from  Klein  (1997)  and 
Dick  & Dick  (1989).  Categories  used  in  previous  reviews  include:  case  studies,  evaluation  reports,  descriptive 
reports,  literature  reviews,  empirical  research,  procedural  how-to,  and  others.  Klein  used  four  categories  to  review 
the  articles  published  in  ETR&D,  the  article  which  provided  the  framework  and  basic  methodology  for  this  review. 
Those  four  categories  were  collapsed,  due  to  the  small  numbers  found  of  empirical  research  and  case  studies,  to 
three  categories  rather  than  four:  empirical  research  (including  case  studies,  which  are  in  fact  a type  of  empirical 
research),  descriptions  or  descriptive  reports  (do  not  contain  data  and  are  not  based  in  the  literature),  and  literature 
reviews. 

Inter-rater  reliability  was  measured  by  comparing  independent  assessments  of  a subset  of  the  articles 
reviewed  for  the  study.  Twenty-five  articles,  including  at  least  one  from  each  category,  were  reviewed  separately  by 
the  authors,  and  the  assessments  compared.  It  was  not  possible  to  review  at  least  one  of  each  type  for  each 
publication  as  two  of  the  publications  were  not  well -distributed  in  terms  of  article  types.  The  authors  agreed  on  23 
of  25,  and  the  inter-rater  reliability  value  was  .92. 

Results 

Table  1 shows  how  many  empirical  studies,  literature  reviews,  descriptive  articles,  and  editorials  were 
published  in  the  four  journals  from  1997-1999.  These  data  show  that  most  of  the  articles  published  in  the  four 
journals  were  descriptive  reports  that  did  not  contain  data.  Articles  that  included  data  based  on  empirical  research 
accounted  for  only  7%  of  all  articles  included  in  this  study.  Most  of  these  empirical  studies  were  published  in  PIQ. 


Table  1 . Numbers  by  types  by  publication: 


Empirical 

Research 

Literature 

Review 

Description  / 
Discussion 

Editorial  / 
Commentary 

Totals 

PIQ 

37 

33 

31 

24 

125 

PI 

14 

11 

173 

80 

278 

T&D 

05 

01 

275 

01 

282 

TTM 

00 

00 

101 

03 

104 

Totals 

56 

45 

580 

108 

789 

Implications  and  considerations 

The  intent  of  this  study  is  to  provide  information  on  the  content  of  the  professional  publications  useful  for  a 
broad  audience,  ranging  from  graduate  students  preparing  for  successful  careers  in  Instructional/Performance 
Technology,  professors  working  to  prepare  tomorrow's  performance  professionals,  as  well  as  current  practitioners 
and  their  managers,  with  information  to  help  them  select  the  professional  publications  that  best  fulfill  their 
development  needs.  Assessing  the  current  status  as  well  as  annual  trends  provides  data  to  those  who  must  determine 
how  to  spend  their,  often  scarce,  resources  for  employee  or  student  professional  development. 
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There  is  no  intent  to  compare  these  publications  to  each  other,  or  to  compare  the  organizations  themselves. 
It  is  important  to  note  that  this  is  not  a call  for  these  organizations  to  publish  the  type  of  scholarly  articles  found  in 
journals  like  ETR&D.  However,  a basic  level  of  quality  is  still  required  in  order  for  these  journals  to  actually 
provide  good  material  for  professional  development,  and  data  (focus  on  results)  is  essential  for  this. 

The  data  as  collected  and  reported  provide  critical  information  for  professionals  in  the  ID/PT/IT  field:  The 
astonishing  lack  of  data  impact  the  credibility  and  respect  of  all  practitioners  in  the  field.  If  performance 
accountability  does  not  start  with  performance  experts,  how  can  it  start  with  anyone  else?  There  are  implications  for 
faculty  in  Instructional  Technology  programs,  because  those  faculty  members  develop  and  teach  curricula  that  must 
includes  robust  courses  and  experience  in  evaluation,  in  order  for  practitioners  to  be  better  trained  to  complete 
evaluations  and  report  data  routinely. 

Conclusion 

Data,  accountability,  and  credibility  in  ID/IT  - practitioners  can  gain  credibility  and  earn  respect  by 
collecting,  responding  to,  and  publishing  the  data  from  projects.  Those  who  demonstrate  this  level  of  discipline  and 
are  willing  to  be  accountable  at  the  same  levels  of  other  professionals  can  begin  the  turn  around. 

As  consumers  of  training  become  better  educated  about  what  training  / instruction  is  and  what  they  should 
expect  from  it,  standards  will  rise  for  practitioners.  Those  who  lack  skills  and  knowledge  necessary  to  routinely 
develop  and  evaluate  performance-based  training  may  find  themselves  in  drastic  career  straits  once  the  customers 
(the  learners  / trainees)  and  performance -owning  managers  begin  to  expect  training  and  performance  interventions 
to  work  well.  Data  is  part  of  the  fundamental  ID  process  for  adding  value;  it  provides  a foundation  for  continuous 
improvement.  The  critical  nature  of  expertise  and  the  power  of  process  is  captured  by  the  following  paragraph: 
Perhaps  the  greatest  strength  of  the  ISD  process  is  the  evolutionary  nature  of  the  prescriptive, 
research-based  model  itself.  While  the  practice  of  ISD  still  retains  the  strengths  of  the  empirical 
evaluation  and  revision  cycles,  to  the  extent  research  and  experience  permit,  it  is  prescriptive.  That 
is,  rather  than  depending  extensively  on  the  test-revision  cycle  to  generate  effective  instruction  in 
an  iterative  manner,  every  attempt  is  made  to  incorporate  research  findings  and  past  experience 
into  the  detailed  procedures  and  supporting  ISD  documentation  to  ensure  that  the  instruction 
developed  comes  as  close  to  the  mark  as  possible  the  first  time.  This  improves  the  validity  of  the 
process  while  also  improving  reliability.  This  has  proven  to  be  a powerful  tool  in  large  scale  ISD. 

In  addition,  as  the  process  provides  more  data  from  the  constant  evaluation  process,  the  procedures 
can  be  continually  improved. 

Copyright  © 1996  Island  MultiMedia.  All  rights  reserved.  Reprint  of  this  article  is  allowed  with 
copyright  acknowledgement.  At  http://www.whidbev.com/frodo/isd.htm. 
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Abstract 

The  purpose  of  this  study  was  to  determine:  a.  how  different  types  of  dynamic  visual  facilitate  the 

achievement  of  specific  types  of  educational  objectives , b.  whether  the  use  of  dynamic  visualization  influenced  the 
amount  of  time  needed  by  learners  to  process  the  information , and  c.  whether  there  is  an  interaction  between  the 
amount  of  time  learners  view  the  animation  and  the  different  levels  of  dynamic  visualization.  Two  hundred  students 
were  randomly  assigned  to  four  treatments \ received  their  respective  instructional  presentation  and  received  four 
individual  criterion  measures.  Results  indicated  that  insignificant  differences  in  achievement  existed  among  the 
visual  treatment  groups  on  all  criterion  measures  and  that  an  insignificant  interaction  was  found  to  exist  between 
time  and  dynamic  visualization. 

Introduction 

Visualization  properly  designed,  and  positioned  in  instructional  presentations  has  been  found  to  be  an 
important  variable  in  facilitating  student  achievement  of  different  types  of  educational  objectives  (Dwyer,  1978). 
However,  the  use  of  visualization  alone,  while  it  can  significantly  improve  achievement,  does  not  always  optimize 
achievement  of  the  more  complex  types  of  learning  outcomes  (Dwyer,  1987).  Static  visuals  in  and  of  themselves 
are  limited  in  their  ability  to  instigate  the  higher  levels  of  information  processing  since  at  best  they  represent 
suggested  motion  attributes  in  dramatic  form  (Reiber  & Kini,  1996).  Park  and  Hopkins  (1993)  have  suggested  that 
dynamic  visual  displays  which  utilize  graphic  movement  in  visualization  to  identify,  interpret  and  reduce  the  level  of 
abstraction  will  facilitate  higher  levels  of  information  processing  and  lead  to  higher  levels  of  achievement.  It  is 
assumed  that  the  static  visuals  and  narration  along  with  increased  animation  would  provide  additive  codes  leading  to 
more  effective  encoding  and  knowledge  transfer  of  the  information  being  presented.  It  was  also  anticipated  that  the 
use  of  animation  (changing  color,  motion,  etc.)  in  static  visuals  would  focus  learning  attention  more  precisely, 
facilitate  focused  interaction  and  thereby  result  in  improved  learning  outcomes.  Additionally,  an  increased 
interaction  with  the  content  materials  would  necessitate  an  increased  amount  of  time  required  for  learners  to  interact 
and  process  the  information  being  presented.  Carroll  (1963)  reports  that  the  amount  of  time  required  would  be 
proportional  to  the  complexity  of  the  visualization,  the  types  of  animation  and  the  level  of  learning  to  be  achieved. 
Specifically,  the  purpose  of  this  study  was  to:  a.  determine  how  different  types  of  dynamic  visuals  facilitate  the 
achievement  of  specific  types  of  educational  objectives,  b.  determine  whether  the  use  of  dynamic  visualization 
influences  the  amount  of  time  needed  by  learners  to  process  the  information  and  c.  determine  whether  there  is  an 
interaction  between  amount  of  time  learners  view  the  animation  and  the  different  levels  of  dynamic  visualization. 

Methodology 

The  2000  word  instructional  unit  used  in  study  was  intended  to  instruct  learners  on  the  nomenclature  of  the 
heart,  and  the  functions  of  the  various  parts  of  the  heart  during  its  systolic  and  diastolic  phases.  The  lesson  included 
1 9 separate  “instructional  frames”  comprised  of  sound  and  graphics  elements.  Each  “frame”  was  accompanied  by 
some  audio  narration  that  described  and  supported  the  visual  being  shown.  In  this  case,  it  identified  and  described 
the  parts  of  the  heart  and  their  functions.  The  lesson  allowed  the  learner  only  one  pathway  (a  linear  progression) 
through  the  lesson,  so  that  the  19  instructional  frames  comprising  the  lesson  were  presented  to  all  students  in  a fixed 
sequence.  Within  each  instructional  frame  there  was  a fixed  sequence  of  audio  and  visual  components. 
Macromedia  Director  was  chosen  for  development  and  delivery  since  there  was  a need  for  the  simultaneous 
presentations  of  audio  narration  and  dynamic  visual  elements.  For  each  of  the  19  frames,  the  appearance  of  the 
graphic  was  followed  by  five  seconds  of  silence,  then  by  the  appropriate  narration.  Following  the  narration  the 
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student  was  given  an  opportunity  for  silent  reflection.  Depending  on  the  timing  treatment,  this  post-narration  time 
was  either  8 seconds  (T,)  or  self-paced,  (T2).  A stop-watch  script  was  written  into  the  instructional  program,  which 
recorded  each  student’s  reflection  time.  The  watch  began  automatically  at  the  end  of  the  narration  for  each  frame. 
Simultaneously,  a button  appeared  in  the  frame  entitled  “Continue,”  which  the  student  could  mouse-click  at  any 
time  to  move  on  to  the  next  frame  of  instruction. 

Prior  to  the  instructional  unit,  the  participants  were  given  a brief  orientation  to  the  study,  which  explained 
its  purpose,  and  informed  them  about  the  use  of  the  silent  time  during  each  frame.  It  was  explained  that:  1)  the 
silent  time  preceding  the  narration  was  designed  to  familiarize  them  with,  and  orient  them  to  the  new  features 
presented  in  the  new  visual,  2)  the  audio  narration,  to  communicate  the  instructional  message,  or  content  for  the 
frame,  and  3)  the  silent  time  following  the  narration,  to  allow  a time  for  reflection  and  rehearsal.  The  participants 
also  were  asked  to  follow  this  strategy  carefully  and  consistently  throughout  the  learning  module.  The  orientation 
also  informed  the  student  about  the  testing  procedure  as  well  as  the  approximate  time  it  would  take  to  complete  the 
experiment.  They  were  asked  to  place  their  complete  focus  on  the  learning  at  hand  and  to  participate  as  fully  as 
possible  in  all  aspects  of  the  learning  and  testing  experience. 


Instructional  Time  (“T”  Variable) 

The  instructional  time  variable  examined  how  enhancements  such  as  dynamic  visualization  in  instruction, 
affected  the  length  of  time  students’  needed  to  “attend  to”  (study)  the  additional  visual  cues,  or  details.  Each  of  the 
dynamic  visual  strategies  added  visual  information  that  the  learner  was  required  to  process.  For  the  purposes  of  this 
study,  pre-narration  time  and  narration  time  remained  constant  across  the  treatments.  Pre-narration  time  - when  a 
new  visual  appeared,  five  seconds  of  silence  was  given  for  students  to  orient  themselves  to  the  components  of  the 
new  visual  before  the  narration  began.  Since  all  subjects  were  given  this  period  of  orientation  prior  to  the  narration, 
it  was  considered  a constant.  Narration  time  - although  the  timing  for  the  narration  varied  from  scene  to  scene, 
depending  on  the  complexity  of  each  explanation,  it  was  identical  from  treatment  to  treatment.  In  other  words,  each 
student  receiving  instruction  spent  an  equal  time  listening  to  the  narrative.  Therefore,  narration  time  was  considered 
a constant.  Post-narration  time  - for  the  purposes  of  this  study,  only  the  period  of  time  that  followed  the  narration 
was  manipulated.  The  students  were  directed  to  use  this  time  to  scan  the  visual  and  to  reflect  upon  the  information 
presented  by  the  narration.  Students  were  randomly  assigned  to  one  of  two  timing  treatments. 

1 . Ti  - Timing  Treatment  #1  - 8 seconds: 

Post-narration  time  equaled  8 seconds  for  all  frames.  Once  the  8 seconds  elapsed,  the 
instructional  program  automatically  advanced  to  the  next  frame  of  instruction. 

2.  T2  — Timing  Treatment  #2  - self  paced: 

Post-narration  time  was  controlled  by  the  learner.  When  the  narration  for  the  frame  was  finished 
a “continue”  button  appeared  on  the  screen.  At  this  point,  the  student  could  study  the  current  visual  for  as  long  as 
needed,  and  click  on  the  button  to  proceed  to  the  next  frame.  For  this  self-paced  treatment,  study  times  were 
recorded  for  each  learner  for  each  frame. 

Treatments  - Visual  Enhancements  (“V”  Variable) 

1.  V j — Still  Graphics  (Control):  The  Control  Group  treatment  contained  19  static  visuals  frames  consisting 
of  colored  line  drawings  of  the  human  heart.  These  simple  illustrations  contained  character-generated 
words  on  the  screen  in  combination  with  arrows  and  labels  that  identified  the  elements  being  presented  in 
the  narration. 

2.  V2  - Progressive  Reveal:  Progressive  Reveal  was  a dynamic  visual  enhancement  that  entailed  a sudden 
color  change  in,  or  a sudden  addition  of  graphic  elements.  Location  of  the  animation  enhancements  in 
each  treatment  was  based  on  item  analysis  conducted  by  Torres  (1990).  A sudden  color  change  was 
intended  to  draw  the  learner’s  attention  (cueing)  to  that  part  of  the  graphic  being  discussed  in  the  narration. 
In  the  heart  presentation,  whenever  an  element  of  the  graphic  is  mentioned,  or  described,  the  viewer’s  eye 
is  drawn  to  that  element  by  a sudden  change  in  color  in  the  element.  The  sudden  additional  of  an  arrow 
during  narration  represented  either  the  direction  of  some  movement  (blood  flow),  or  of  some  hidden 
pressure  that  occurred  in  the  heart.  The  Progressive  Reveal  is  intended  to  help  the  learner  locate  the 
important  elements  within  the  whole,  and  to  ignore  the  extraneous  detail.  Progressive  Reveal  was 
intended  to  improve  student’s  abilities  on  identification,  terminology  and  comprehension  test  items  that 
require  students  to  recall  names  of  parts  or  functions. 
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3.  V3  - Animation:  Animation  was  a dynamic  visual  enhancement  that  generated  graphic  motion  simulating 
the  mechanics  of  the  real  world,  such  as  the  flow  of  blood,  or  motions  of  the  valves,  muscles,  and  other 
moving  elements  of  the  heart.  This  dynamic  visual  display  added  the  element  of  motion,  which  was 
intended  to  graphically  demonstrate  the  movement  of  parts  or  the  flow  of  blood  through  the  heart,  as  it 
was  being  described  in  the  audio  narrative.  Animation  was  intended  to  promote  learning  at  the  conceptual 
and  rule  level.  The  animation  treatment  would  most  benefit  students  on  terminology  and  comprehension 
questions  that  recalled  information  related  to  motions,  operations,  processes,  functions  and  how  they  were 
interrelated. 

4.  V4  — Animation  and  Progressive  Reveal:  This  treatment  combined  the  elements  of  Progressive  Reveal  and 
Animation  described  above.  This  combination  of  dynamic  visual  elements  attempted  to  focus  students’ 
attention  on  relevant  details  and  on  the  movements,  operations,  processes,  and  functions  inherent  in  the 
elements  being  described  by  the  audio  narrative.  When  used,  this  combination  of  methods  was  intended  to 
give  the  student  a more  comprehensive  understanding  of  the  interrelationships  between  the  parts  and 
functions  of  the  heart. 

Each  enhancement  described  above  was  assigned  to  a frame  on  the  basis  of  its  value  for  supporting  certain 
instructional  objectives  articulated  in  the  narration  for  that  frame.  In  other  words,  the  Progressive  Reveal  treatment 
only  was  used  when  the  narration  identified,  described,  or  named  parts  or  functions  of  the  heart,  while  the 
Animation  treatment  only  was  used  when  the  narration  described  some  movement,  flow,  or  process. 


Table  1 . Summary  of  Visual  Enhancement  Strategies 


Treatment  V|  (control) 

Narration,  Color  Heart  Drawings,  Labels 

Treatment  V2 

Treatment,  Color  Heart  Drawings,  Labels,  and 
Progressive  Reveal 

Treatment  V3 

Narration,  Color  Heart  Drawing,  Labels,  and 
Animation 

Treatment  V4 

Narration,  Color  Heart  Drawings,  Labels, 
Animation  and  Progressive  Reveal 

During  the  presentation  of  one  of  the  three  visually  enhanced  treatments,  the  selection  and  use  of  that 
particular  visual  enhancement  strategy  was  synchronized  with  the  audio  narration  during  any  instructional  frame,  or 
any  portion  of  a frame,  according  to  the  following  criteria: 

Throughout  Control  Treatment  V!,  still  graphic  images  were  employed  throughout  the  presentation.  A total 
of  19  images  were  employed.  For  Frame  #18,  which  described  the  Systolic  Phase  (in  which  the  heart  exhibits  two 
distinct  movements),  two  still  images  were  presented  consecutively.  In  treatments  V2  and  V4,  the  still  graphics  were 
enhanced  by  the  addition  of  Progressive  Reveal.  Whenever  parts  of  the  heart  were  being  introduced,  described,  or 
identified,  the  parts  and  their  labels  would  suddenly  change  color  to  attract  the  learner’s  attention  to  those  elements. 
In  Treatment  V2  and  V4,  arrows  appeared  as  needed  to  show  the  direction  of  some  movement  or  force.  During 
Treatments  V3  and  V4,  animation  was  used  only  during  those  frames,  or  potions  of  frames  that  describe  motion, 
including  the  contraction  of  the  heart,  the  movement  of  internal  parts  of  the  heart,  or  the  flow  of  blood  from  chamber 
to  chamber.  During  Treatment  V4,  both  still  and  animated  frames  were  enhanced  by  the  addition  of  Progressive 
Reveal  as  needed  when  the  narrative  introduced,  described  or  identified  parts  or  areas  of  the  heart.  As  in  Treatment 
V2,  this  enhancement  involved  a sudden  color  change  for  the  part  and  for  its  corresponding  label. 

Criterion  Measures 

The  dependent  variables  used  in  this  study  were  the  achievement  levels  of  students  on  specific  learning 
objectives.  Achievement  of  the  specific  learning  objectives  was  determined  by  the  following  criterion  measures 
(Dwyer,  1978,45-47) 

Drawing  Test  (20  items):  This  test  consisted  of  a numbered  list  of  twenty  items  (naming  the  parts  of  the 
heart)  depicted  in  the  instructional  unit.  The  students  were  asked  to  draw  a reasonable  facsimile  of  the  human  heart, 
and  to  place  the  number  corresponding  to  each  item  in  its  correct  location  on  the  drawing.  Emphasis  was  placed  on 
the  accurate  positioning  of  the  numbered  items,  not  on  the  quality  of  the  drawing.  The  drawing  test  was  intended  to 
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assess  the  students’  ability  to  reproduce  or  reconstruct  the  heart  and  to  maintain  the  appropriate  special  and 
contextual  relationship  between  and  among  the  various  parts. 

Terminology  Test  (20  items):  Given  five  choices  in  each  item  in  a multiple  choice  test,  learners  were 
required  to  select  the  part  or  function  of  the  heart  described  in  each  item.  This  test  was  designed  to  measure  a 
student’s  ability  to  choose  the  appropriate  graphic  representation  of  a part  or  function,  given  an  abstract  definition  or 
description.  This  related  to  knowledge  of  facts,  terms,  and  definitions,  which  would  be  needed  as  prerequisites  for 
the  learning  of  concepts,  rules,  and  principles,  the  higher  intellectual  skills,  which  would  entail  more  complex  levels 
of  learning  abstraction. 

Comprehension  test  (20  items):  Each  question  described  the  activity  in  a certain  part  of  the  heart,  at  a 
single  moment  in  the  cycle  of  the  heart’s  movements.  It  required  students  to  show  how  another  part  (or  parts)  would 
appear  at  that  same  moment  by  selecting  an  illustration  which  most  accurately  represented  that  condition.  This  test 
required  students  to  have  a thorough  understanding  of  the  heart,  its  parts,  its  sequence  of  movements  and  cycles,  its 
simultaneous  processes,  and  the  relationships  between  and  among  the  parts  of  the  heart  at  critical  moments  during 
its  cycle  of  activity.  This  test  was  intended  to  measure  a students’  ability  to  apply  the  facts,  concepts,  rules,  and 
processes  presented  in  the  instruction. 

In  the  case  of  the  Drawing  Test,  Dwyer’s  original  version  (1965)  was  used.  The  visual  version  of  the 
multiple  choice  tests  for  identification,  terminology,  and  comprehension  were  taken  from  Dwyer  (1985).  In  each 
case,  the  item  stems  were  textual,  while  the  options  were  visual  (line  drawings). 

Total  Comprehension  Measure  (80  items):  This  score  was  obtained  by  adding  the  scores  for  all  four  tests 
into  a total  test  score.  It  was  intended  to  provide  a comprehensive  assessment  of  the  learner’s  overall  understanding 
of  the  content  presented. 

Experimental  Design 

The  study  utilized  a randomized,  post  test  only,  control  group  design.  Each  of  the  four  visual  treatments 
was  tested  against  the  two  timing  patterns  in  a 4 (visualization)  x 2 (timing)  mixed  factorial  design  (Campbell  & 
Stanley,  1966).  Each  of  the  eight  treatments  were  subjected  to  five  separate  post-tests,  or  dependent  measures.  In 
Table  3,  the  post-tests  are  represented  by  0i,  02,  03,  04,  05,  which  refer  to  Drawing,  Identification,  Terminology, 
Comprehension,  and  Total  Comprehension  tests,  respectively. 

Table  2.  Mixed  Factorial  Design 


Timing  (T) 

8-Second  (Tl) 

Self-Paced  (T2) 

Visualization  (V) 

V|T, 

v,t2 

Still  Images  - Control  (VO 

V2T, 

v2t2 

Animation  (V2) 

VjT, 

v3t2 

Graphic  Reveal  (V3) 

V3T, 

v3t2 

Animation  and  Graphic  Reveal  (V4) 

V4T, 

v4t2 

Table  3.  Combinations  of  Treatments  and  Post-Tests 


Tests  (0) 

Treatments  (V&T) 

Drawing  (Oi) 

Ident.  (02) 

Termin.  (O3) 

Comp.  (04) 

Total  Comp  (05) 

V,T, 

V.T.O, 

v,t,o2 

v,t,o3 

v,t,o4 

V.TA 

v2T, 

v2T,o, 

V2T102 

v2t,03 

v2t,o4 

v2T,o5 

V3T, 

v3t,0, 

v3T,o2 

v3t,o3 

v3t,04 

v3t,os 

V4T, 

v4t,o, 

v4t,o2 

v4t,o3 

v4t,o4 

v4ta 

v,t2 

v,t2o, 

v,t202 

v,t2o3 

v,t2o4 

V.TA 

v2t2 

v2t2o, 

V2T202 

v2t2o3 

v2t2o4 

v2TA 

v3t2 

v3t2o, 

v3t202 

v3t2o3 

v3t2o4 

VjTA 

v4t2 

v4t2o, 

v4t2o2 

v4t2o3 

v4t2o4 

v4TA 
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Result  and  Discussion 


Table  4.  Mean  Scores  Achieved  by  Students  on  Each  of  the  Criterion  Tests 


Draw. 

Ident. 

Term. 

Comp. 

Total  Comp. 

Group 

Treatment 

N 

Mean 

Mean 

Mean 

Mean 

Mean 

VI 

Static 

50 

17.50 

17.72 

14.06 

13.16 

62.44 

V2 

Progressive  Reveal 

50 

15.78 

17.40 

13.20 

12.76 

59.14 

V3 

Animated 

50 

16.78 

17.80 

13.42 

12.62 

60.62 

V3 

Combined 

50 

16.12 

16.96 

12.90 

12.26 

58.24 

ANOVAs  conducted  on  the  individual  and  total  criterion  measures  indicated  insignificant  differences  in 
achievement  among  the  visual  treatments.  Significant  differences  in  favor  of  the  self-paced  treatments  were  found 
to  exist  on  the  Drawing  (F=6.81,  df=l/2,  p.<  05)  and  total  Criterion  Measures  (F=4.59,  df=l/2,  p.<  05). 

Table  5 presents  the  study  times  interacted  with  their  respective  treatments.  Insignificant  interactions  were 
found  to  exist  between  the  visual  treatments  and  time  on  all  criterion  measure. 

Table  5.  Study  Times  (Means) 


Group 

Treatment 

T]  8-Second 

T2  Self-Paced 

V, 

Static 

152  sec. 

173  sec. 

V2 

Progressive  Reveal 

1 52  sec. 

190  sec. 

V3 

Animated 

1 52  sec. 

249  sec. 

V4 

Combined 

152  sec. 

170  sec. 

The  results  of  this  study  indicated  that  the  types  of  animation  strategies  employed  provided  insignificant 
differences  in  student  achievement  on  the  types  of  criterion  measures  measuring  achievement  of  different  types  of 
educational  objectives.  The  study  also  found  that  timed  sequences  were  most  efficient  in  terms  of  amount  of  time 
students  spent  interacting  with  the  visualized  treatments.  These  results  implied  that,  at  least  for  this  group  of 
learners,  static  visualization  was  sufficient  to  facilitate  learning.  This  contradicted  the  current  ideas  and  trends 
concerning  the  selection  and  use  of  animation  in  educational  and  learning  environments. 

A number  of  possible  explanations  may  be  suggested:  (a)  students  were  not  properly  oriented  to  the 
importance  of  the  animation  and  thereby  were  not  prepared  to  profit  from  the  information  being  provided,  (b)  the 
experimental  environment,  itself,  motivated  all  students,  so  that  the  control  students  interacting  with  the  static 
visuals  achieved  as  well  as  the  students  receiving  the  animated  visualization,  (c)  the  animation  employed  may  not 
have  been  sufficiently  intense  so  as  to  instigate  the  levels  of  information  processing  necessary  to  move  the 
information  from  short  term  into  long  term  memory.  Finally,  (d)  the  high  cognitive  abilities  of  the  students  involved 
may  have  pre-empted  the  need  for  dynamic  visual  support.  Consequently,  the  students  may  not  have  worked 
conscientiously  to  integrate  the  animated  information  into  higher  levels  of  cognitive  attainment  (Rieber  & Keni, 
1991).  In  actuality  their  attention  to  the  dynamic  visualization  may  have  impeded  rather  than  facilitated  the  required 
levels  of  information  processing. 

It  is  also  important  to  report  that  studies  conducted  by  Lumsdaine,  Sultzer,  and  Kopstein  (1961)  and  May  & 
Lumsdaine  (1958)  have  indicated  that  animation,  which  is  being  used  primarily  to  enhance  the  attention  focusing  or 
realism  of  a presentation,  does  not  have  a significant  effect  on  learning.  Anglin,  Towers, and  Levie  (1996)  have 
indicated  that  some  progress  is  being  made  on  visualization  in  facilitating  knowledge  acquisition,  knowledge 
integration,  and  knowledge  generation. 

Considering  the  relationship  between  cost  of  producing  different  types  of  graphics  and  their  relative 
effectiveness  in  promoting  learning,  this  study  demonstrated  that,  in  some  cases,  still  images  can  be  as  effective  as 
more-costly  animations.  Hopefully,  this  will  stimulate  additional  work  into  this  important  area  of  research. 
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Abstract 

Online  computer  conferences  used  to  assist  distance  learning  courses  often  fail  because  the  moderator — 
usually  the  instructor  responsible  for  the  conference — is  not  properly  trained  in  techniques  that  build  a community 
of  learners.  It  has  often  been  assumed  that  the  skills  required  to  create  a vibrant  classroom  discussion  translated 
easily  to  an  online  forum.  This  has  been  rarely  the  case. 

This  study  utilized  the  qualitative  methods  of  grounded  theory  and  narrative  research  to  explore  how  a 
moderator , after  undergoing  training , would  affect  students  in  one  of  three  segregated  computer  conferences 
supporting  an  online  course.  The  training  was  based  on  both  the  academic  literature  on  educational  computer 
conference  moderation  and  situational  examples  taken  from  the  experiences  of  online  moderators.  The  students  * 
experiences  with  the  trained  moderator  were  compared  with  those  students  in  the  other  two  computer  conferences 
without  a trained  moderator.  The  data  analyzed  were  comprised  of  the  messages  collected  from  the  three  computer 
conferences , selected  interviews , extensive  journals  written  by  the  researcher , and  an  online  survey . 

The  study  also  considered  the  problems  and  pressures  stemming  from  unclear  policies  for  constructing  an 
online  course  in  an  environment  of  overlapping  departmental  mandates.  These  mandates  resulted  in  more  emphasis 
being  given  to  putting  courses  online  than  the  choice  of  the  most  appropriate  pedagogy. 

The  results  indicated  that  a trained  moderator  had  a positive  effect  on  computer  conferences  as  a community  of 
support  and  warmth  was  built;  while  another  group,  without  such  a moderator , constructed  a community  based  on 
group  dissent.  No  community  of  any  sort  was  found  in  the  third  group. 

Overview  of  the  Problem 

The  rush  to  get  online  can  find  teachers  in  the  position  of  having  an  excellent  command  of  classroom 
pedagogy  but  without  a full  complement  of  the  skills  needed  to  make  the  most  of  the  technologies  and  conventions 
of  online  distance  learning.  It  is  not  unusual  for  there  to  be  no  clear  policy  to  educate  teachers  in  this  regard. 
Frequently  asynchronous  Internet  courses  contain  a computer  conference  wherein  the  class  can  communicate  with 
each  other.  A computer  conference  is  used  as  an  online  equivalent  of  classroom  discussion,  a place  where  students 
can  participate  in  collaborative  work  and  carry  on  other  activities.  Historically,  the  moderating  of  such  conferences 
have  been  the  responsibility  of  the  teacher  or  assigned  to  a student  to  manage  (Murphy,  Cifuentes,  Yakimovicz, 
Segur,  Mahoney,  & Kodali,  1996).  An  incorrect  but  frequently  made  assumption  is  that  the  ability  to  lead  an 
exhilarating  classroom  discussion  translates  well  to  leading  an  online  discussion.  A common  occurrence  in  computer 
conferencing  is  that  the  teacher  poses  a few  questions  to  the  group,  and  no  one  responds.  Often,  the  conference 
becomes  unused  to  the  bewilderment  of  the  teacher.  Managing  interaction  in  a distance  learning  environment 
requires  different  skills  than  in  a traditional  classroom  (Inman  & Kerwin,  1999). 

It  is  widely  assumed  that  students  will  use  an  online  conferencing  system  just  because  it  is  available.  This 
is  not  the  case.  The  Apple  Classroom  of  Tomorrow  study  put  computers  in  schools  for  ten  years  as  an  exploratory 
experiment.  Researchers  believed  that  merely  having  technology  available  would  increase  learning.  They  were 
disappointed  to  find  that  it  did  not.  Instead,  they  found  that  technology,  by  itself,  does  nothing.  Rather  than  being  a 
panacea,  technology  should  be  regarded  as  nothing  more  than  a tool  that  has  the  potential  to  enhance  innovative 
thinking  (Sandholtz,  Ringstaff,  & Dwyer,  1997).  It  is  the  same  with  Internet  based  instruction. 

Students  often  experience  isolation  due  to  the  lack  of  the  usual  social  aspects  of  a classroom  (Ahem,  1995). 
The  immediacy  of  interactions  between  instructor  and  student  as  well  as  among  students  is  diminished  because 
asynchronous  communication  can  take  days  or  even  weeks  between  the  time  a question  is  asked  or  a point  is  made, 
and  the  student  receiving  feedback.  This  time  lag  can  leave  students  feeling  ignored  (Eastmond,  1995).  Attrition  in 
distance  learning  courses  is  higher  than  their  face-to-face  counterparts,  sometimes  much  higher  (Carr,  2000;  Cheng, 
Lehman,  & Armstrong,  1991;  Jewett,  1997;  Noble,  1998).  Studies  report  that  completion  rates  of  students  in 
traditional  face-to-face  classes  can  be  as  high  as  60%  greater  than  online  classes.  This  determined  that  the  use  of 
trained  moderators  did  alleviate  a number  of  these  problems. 
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Intent  of  the  Study 

The  intent  of  the  study  was  to  explore  how  the  use  of  a trained  moderator  affected  both  the  students  and 
instructor  of  an  Internet  based  distance  learning  class.  The  moderator  underwent  training  based  on  academic 
literature  on  educational  computer  conference  moderation  and  examples  taken  from  the  experiences  of  online 
moderators. 

Limitations  of  the  Study 

The  limitations  of  this  study  were  broad  and  affected  the  generalizability  of  its  findings.  The  study 
involved  one  course  given  over  one  semester.  The  students  involved  were  unique  individuals  each  taking  the  course 
for  different  reasons.  The  woman  that  volunteered  to  be  the  trained  moderator  was  under  a set  of  pressures  unique 
to  her  particular  circumstance.  The  instructor  utilized  one  form  of  Internet  based  distance  learning  that  had  strong 
similarities  to  a correspondence  course.  There  are  many  other  ways  of  constructing  a distance  learning  course.  The 
specificity  of  what  transpired  in  the  course  had  a strong  bearing  on  the  generalizability  of  the  findings  of  the  study. 
Qualitative  research  does  not  purport  to  generate  universally  generalizable  findings,  but  rather  to  discover  and 
uncover  a set  of  observations,  processes,  and  theories  that  other  researchers  can  transfer  to  similar  research  problems 
in  other  situations  and  fields  (Charmaz,  2000;  Eisner,  1997). 

Writing  Myself  In 

In  an  interpretive  study,  such  as  this,  all  aspects  of  the  study  are  filtered  through  the  eyes  of  the  researcher, 
which  in  this  case  was  myself  Therefore,  it  has  been  common  practice  to  make  the  interpretive  process  explicit 
(Bailey,  1996).  Instead  of  seeming  to  be  a neutral  observer  without  bias,  Piantanida  and  Garman  (1999)  suggested 
that  it  was  incumbent  upon  the  researcher  to  expose  biases  of  the  researcher  by  discussing  why  the  study  was 
personally  compelling.  Along  with  disclosure  of  the  research  process,  personal  disclosure  helps  the  reader  gain  an 
understanding  of  my  point  of  view  and  why  certain  decisions  and  interpretations  were  made.  A discussion  of  what 
brought  me  to  this  work  clarifies  my  point  of  view  as  it  affected  this  study. 

I have  been  online  since  the  early  1980s  when  I bought  my  first  computer,  an  Apple  ][+  along  with  a 300 
baud  modem.  At  first,  what  I found  was  not  only  disheartening,  it  was  also  boring.  In  northern  New  Jersey,  the  BBS 
world  was  inundated  with  vanity  and  inanity.  Boards  were  mostly  run  by  spoiled  upper  middle-class  boys  between 
the  ages  of  14  and  16  who  used  the  venue  to  pump  up  their  egos.  The  systems  were  rife  with  the  collection  and 
transfer  of  illegal  MCI  telephone  card  codes. 

The  idea  occurred  to  me  that  a BBS  could  be  a warm  and  friendly  place,  appealing  to  a different  group  of 
people  than  kids  skirting  the  law.  I developed  the  idea  of  a digital  restaurant  where  people  could  visit  and  discuss 
the  news  of  the  day  or  anything  else  they  had  in  mind,  in  a supportive,  non-threatening  environment.  There  would 
be  no  flames  (personal  attacks)  nor  would  there  be  pirating  of  software.  I felt  that  my  bulletin  board  should  reach 
out  to  the  user,  not  the  other  way  around,  and  attempt  to  be  an  oasis  of  intelligence  in  the  suburbs.  I bought  some 
BBS  software  and  learned  how  to  twist  and  turn  it  into  a restaurant  named  *+  DAVID'S  PLACE  +*. 

When  a user  logged  on,  he  or  she  was  greeted  at  the  door  by  a loquacious  Maitre’d  who  profusely 
welcomed  the  new  user.  The  user,  who  was  referred  to  as  a guest,  was  met  with  a description  of  the  restaurant  in 
paragraphs  of  purple  prose  that  included  everything  from  the  pile  of  the  carpet  to  the  richness  of  the  oak  walls.  The 
guest  was  then  seated  at  a table  and  proffered  the  menu,  which  contained  the  options  that  were  available  for 
ordering.  These,  naturally  enough,  were  the  features  of  the  BBS  software.  I was  Chef  David,  who  bounded  out  of 
the  kitchen  and  again  welcomed  the  guest.  My  persona  was  written  into  each  prompt  and  instead  of  the  usual 
unfriendly  response  of  ? or  SYNTAX  ERROR,  the  software,  when  it  did  not  understand  a response  would  display 
something  along  the  order  of, 

I APOLOGIZE  BUT  THE  BROWNIES  JUST  BURNED  AND  THE  SMOKE  FROM  THE 

OVEN  GOT  IN  MY  EYES  SO  I COULDN’T  MAKE  OUT  WHAT  YOU  WERE  SAYING. 

WOULD  YOU  REPHRASE  THAT? 

I advertised  the  BBS  on  some  other  BBSs  and  waited  for  the  phone  to  ring.  I did  not  wait  very  long.  In  a 
matter  of  weeks,  word  somehow  got  out  and  I had  to  devote  my  Apple  ][+  to  the  BBS  and  buy  myself  a second 
computer  because  +*  DAVID’S  PLACE  +*  quickly  became  busy  around  the  clock.  A core  group  of  guests  were 
formed,  and  after  being  prompted  a bit,  engaged  in  discussions  on  a fairly  high  plane.  The  group  quickly  coalesced 
into  a kind  of  round-table  that  did  not  easily  suffer  idiots. 
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I learned  that  telecommunication  was  a wonderful  medium  for  building  powers  of  persuasion,  organization, 
and  thought.  I found  that  when  one  wrote  a message  it  tended  to  be  well  composed  because  the  process  of  thinking, 
typing,  and  reading  turned  people  into  editors  who  reflected  on  their  thoughts  before  making  them  public.  There 
was  also  an  implicit  standard  of  quality  that  was  not  taken  lightly  by  the  guests.  My  BBS  gave  people  the 
opportunity  to  use  telecommunication  in  a supportive  and  mentally  stimulating  atmosphere.  Frequenting  the  BBS 
taught  many  people  who  previously  were  unfocused  and  disorganized  in  their  writing,  how  to  write  and  write  well. 
Through  the  use  of  my  BBS,  I gave  many  people  a reason  to  become  interested  in  computers  and  who,  years  later, 
wound  up  with  computer  science  degrees  and  jobs  in  the  computer  industry. 

Finding  ways  to  manage  personal  conflicts  and  heated  arguments  among  guests  did  not  come  easily  or 
naturally.  I found  myself  experimenting  with  a combination  of  private  email  and  publicly  written  messages  to  douse 
verbal  flames,  as  some  of  the  stronger,  quicker,  but  painfully  antisocial  writers  tried  to  rout  other  guests.  This  was 
my  first  taste  of  moderating  online  conferences,  I went  about  it  totally  through  trial  and  error.  Although  there  were 
times  I came  close  to  just  turning  off  the  modem,  I considered  it  a worthy  challenge.  The  experience  taught  me 
about  using  language  to  manipulate.  Manipulate  is  a charged  word.  According  to  the  Merriam- Webster  dictionary 
(1998)  it  can  mean,  “a):  to  manage  or  utilize  skillfully”  or  “ b ) to  control  or  play  upon  by  artful,  unfair,  or  insidious 
means  esp.  to  one’s  own  advantage”  (p.  708).  I preferred  the  former  and  used  the  word  in  that  positive  context  in 
this  study. 

I decided  to  shut  it  down  when  it  started  to  feel  like  more  work  than  play  and  quickly  found  myself  on  the 
CompuServe  Information  Service’s  MAUG  (Micronetted  Apple  User  Group)  forums.  Eventually  I became  SysOp 
(System  Operator)  of  the  Macintosh  Community  Clubhouse  Forum  (MACCLUB).  This  forum  was  languishing  and 
had  nothing  to  recommend  itself  aside  from  being  the  place  that  housed  the  classified  advertisements.  It  logged  an 
average  of  seven  or  eight  messages  per  day.  I saw  it  as  an  opportunity  to  reincarnate  *+  DAVID’S  PLACE  +*  on  a 
larger  scale,  so  I restructured  the  forum  by  getting  rid  of  the  dusty  ill  used  message  topic  sections  and  libraries  and 
filled  it  with  things  that  excited  me  and  that  I felt  would  excite  the  Macintosh  community. 

After  a few  weeks,  an  average  day  brought  over  100  posted  messages.  After  a few  months  a typical  day 
brought  250  messages  which,  during  periods  of  heat  and  passion,  grew  to  over  500  postings  daily.  This  was  a very 
creative  and  exhilarating  time  for  me.  I quickly  learned  what  worked  and  was  quick  to  change  anything  that  did  not 
work.  Under  the  basic  rule,  it  was  allowable  to  attack  ideas  posted  in  a message  but  it  was  not  allowable  to  attack 
the  person  who  wrote  the  message.  This  basic  rule  prevented  many  online  flames  but  when  a flame  did  occur,  it 
took  creativity  and  wit  to  quell  it  without  resorting  to  banishing  the  offending  writer  from  the  community.  No  one 
was  ever  sent  away.  My  15  years  of  involvement  online  taught  me  much  about  what  builds  or  destroys  online 
communities. 

I decided  to  rejoin  academia  and  began  to  read  the  research  on  educational  computer  conferencing  written 
over  the  same  15-year  period.  I was  not  surprised  to  find  that  most  researchers  had  discovered  exactly  the  same 
concepts  through  research  that  I had  found  through  practicing  my  hobby.  After  returning  to  academia  I soon 
realized  that  I did  not  have  the  time  to  devote  to  the  community,  so  I ceased  being  its  moderator. 

Most  people  who  have  moderated  online  conferences  learned  by  observing  other  moderators  and  then 
through  the  trial  and  error  of  doing  it  (Berge  & Collins,  1998;  Collins  & Berge,  1997).  At  the  start  of  my  doctoral 
work,  a vague  and  shapeless  idea  occurred  to  me  that  learning  how  to  moderate  by  trial  and  error  was  very 
inefficient  and  that  the  skills  of  a moderator  were  something  that  could  be  taught.  As  I progressed  through  my 
program  of  study  and  learned  more  about  distance  learning  in  the  context  of  Internet  based  delivery  and  computer 
conferencing  my  ideas  clarified  and  resulted  in  this  study. 

Research  Procedures 

Qualitative  Research 

This  study  used  a variety  of  qualitative  analysis  techniques  to  attempt  to  clarify  a series  of  situations  that, 
during  data  collection  and  analysis,  widened  in  scope  and  complexity  as  new  issues  arose  and  became  integral  to  the 
story.  Miles  and  Huberman  (1994)  maintained  that  qualitative  research  is  more  of  a craft  than  a set  of  rules  and  that 
no  study  conforms  perfectly  to  a predetermined  methodology.  Any  methods  that  afford  clear  and  “credible  meaning 
from  a set  of  qualitative  data  is  grist  for  our  mill  regardless  of  its  antecedents”  (p.  3). 

Attempting  to  define  qualitative  research  is  a bit  like  trying  to  describe  smoke.  It  seems  easier  to  explain 
what  it  is  not  than  what  it  is.  According  to  Savenye  and  Robinson  (1996)  it  is  marked  by  rich  and  detailed 
descriptions  of  the  behaviors  of  people  who,  in  their  actions,  construct  their  own  realities  that  influence  the  meaning 
of  their  actions.  It  questions  just  what  is  going  on  and  what  variations  can  be  found  in  the  phenomenon  being 
examined  (Lofland,  1971). 
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Participants  and  Conference 

The  students  were  30  upper  level  undergraduates  enrolled  in  ICJ-450,  Implications  of  Supreme  Court  Decisions  on 
Law  Enforcement  at  a large  Southwestern  University.  The  gender  distribution  of  the  class  was  21  women  and  9 
men.  I decided  to  break  the  class  into  three  conferences  that  were  called  law  firms  due  to  the  context  of  the  subject 
matter.  Each  conference  was  restricted  to  reading  and  responding  to  messages  within  the  particular  conference 
assigned  to  the  student.  The  initial  design  called  for  one  conference  to  be  lead  for  the  entire  semester,  by  a 
moderator  who  would  undergo  training  consisting  of  both  face-to-face  training  and  a written  job  aid  of  my  design 
For  the  other  two  conferences,  one  student  each  week  would  be  assigned  the  job  of  moderating.  Due  to  the 
Instructor  failing  to  communicate  with  students  regarding  moderating  the  two  conferences  without  a trained 
moderator,  most  weeks,  any  moderating  activities  in  these  conferences  was  ad-hoc.  The  class  used  the  FirstClass 
conferencing  system. 

Structure  of  the  Course 

The  way  that  Stanley  Pike,  the  Instructor,  structured  the  course  was  not  very  conducive  to  interaction. 
Each  week  the  students  were  required  to  read  a number  of  supreme  court  decisions,  answer  questions  about  them 
and  send  the  results  back  to  the  Pike  who  would  send  a one  line  email  detailing  it  the  work  was  sufficient  or  needed 
to  be  redone.  There  was  no  student-student  interaction  built  into  the  course  and  the  student-teacher  interaction  was 
quite  minimal.  The  course  was  conducted  via  distance  once  before  I got  involved  and  the  conferencing,  although 
there  was  presented  as  something  for  the  students  to  use  if  they  chose.  The  messages  may  or  may  not  be  read  by 
Stanley.  The  result  was  a total  of  four  messages  written  over  the  entire  semester  from  a class  of  approximately  30 
students. 

Data  Collection 

Data  were  collected  from  763  online  messages,  which  were  written  by  the  members  of  the  three  segregated 
computer  conferences.  My  role  was  that  of  an  observer.  Throughout  the  study  I wrote  detailed  memos  explaining 
the  process  and  my  perspectives  on  what  had  transpired.  Detailed  coding  memos  were  also  written  as  the  data  were 
analyzed.  Data  were  also  provided  by  email  and  phone  conversations  between  myself  and  the  trained  moderator; 
Rob  Janesh  from  the  College  of  Distributed  Education;  Barbara  Malik,  the  technology  support  analyst  from  the 
School  of  Legal  Services;  and  Stanley  Pike,  JD,  the  instructor  of  the  course.  After  the  end  of  the  course  I conducted 
audio  taped  interviews  with  each  of  these  people;  the  transcription  of  the  tapes  added  to  the  data.  Near  the  end  of 
the  course,  an  online  survey  was  completed  by  22  of  the  30  students  who  finished  the  course. 

Data  Analysis 

I used  a combination  of  grounded  theory  and  narrative  analysis  to  interpret  and  describe  the  data.  The  thrust 
of  grounded  theory  is  toward  developing  theory  without  regard  to  the  type  of  data,  or  lines  of  research,  which  makes 
it  not  a method  or  technique,  per  se.  Instead,  it  is  a “style”  of  engaging  in  qualitative  research  that  includes  a 
number  of  distinct  features  and  guidelines,  such  as  using  constant  comparisons  and  a paradigm  of  coding  to  ensure 
the  development  of  concepts  (Strauss,  1987).  In  assessing  a grounded  theory,  the  research  process  must  be 
explicated  and  conform  to  the  rather  broad  requirements  of  the  constant  comparative  method  such  as  engaging  in 
comparisons  between  data  as  soon  as  data  is  collected.  For  the  study  to  be  considered  adequate,  the  resulting  theory 
should  fit  the  phenomenon  studied  and  be  general  enough  to  broadly  cover  a range  of  situations  (Wells,  1995). 

Although  grounded  theory  began  as  a postpositivist  mode  of  inquiry,  when  it  is  subjective  and  relativist, 
grounded  theory  can  evolve  to  fit  a constructivist  perspective  (Annells,  1996).  Charmaz  (2000)  proposed  a 
constructivist  approach  to  grounded  theory  that  does  not  attempt  to  find  truth  that  can  be  generalized,  instead  its  use 
is  to  generate  concepts  that  can  be  transported  to  similar  problems  in  other  fields.  An  interpretivist  mode  of 
grounded  theory  was  used  in  this  study.  To  assist  me  the  in  massive  task  of  analyzing  763  text  messages  posted  to 
the  computer  conferences,  I coded  useing  the  software  program  QSR  NUD*IST.  The  acronym  stands  for  Non- 
numerical  Unstructured  Data  Indexing  Searching  and  Theorizing  (Qualitative  Solutions  and  Research  Pty  Ltd., 
1997).  NUD*IST  is  a program  designed  for  the  flexible  storage,  coding,  retrieval,  and  analysis  of  text.  Weitzman 
and  Miles  (1995)  declared  that  it  is  one  of  the  best  software  options  available.  They  contend  that  the  developers  did 
a masterful  job  of  determining  and  providing  for  the  actions  and  features  required  for  a wide  variety  of  text  analyses. 
After  a sizable  number  of  iterations  of  coding  one  major  criteria  emerged:  'What  built  or  destroyed  community'. 

Narrative  research  refers  to  any  study  that  analyzes  narrative  materials  (Lieblich,  Tuval-Mashiach,  & 
Zilber,  1998).  It  is  a form  of  qualitative  analysis  using  plot  and  stories  to  describe  events  and  situations  and  has  a 
variety  of  meanings  and  procedures  dealing  with  how  protagonists  interpret  things  (Bruner,  1990;  Riessman,  1993). 


465 


472 


Polkinghome  (1995)  divided  narrative  research  into  two  classes:  paradigmatic  and  narrative  construction.  Narrative 
construction  is  not  merely  a retelling  of  the  actions  and  thoughts  of  the  protagonist;  it  attempts  to  bring  meaning  and 
significance  to  them.  The  researcher,  in  this  form  of  narrative  inquiry,  is  the  narrator  who  tells  the  story  in  his  or  her 
own  voice.  The  paradigmatic  type  of  narrative  research  produces  knowledge  of  concepts  that  classify  instances  to 
categories  with  shared  common  attributes.  Paradigmatic  narrative  research  has  apparent  similarities  to  grounded 
theory.  Often  referred  to  as  analysis  of  narrative,  the  researcher  inspects  a number  of  stories  to  discover  connections 
and  similarities  between  them. 

The  presentation  of  analysis  of  narrative  often  takes  the  form  of  extended  quotations  from  unedited  data, 
along  with  the  stories  leading  up  to  the  data,  and  attempts  to  analyze  what  took  place  through  the  perspectives  of  the 
commonality  emergent  from  the  data  under  study  and  varied  other  data.  This  study  uses  both  types  of  narrative. 

Struggling  Toward  the  Firms 

The  course  almost  didn't  get  offered  due  to  serious  medical  problems  experienced  by  the  Instructor  which 
prevented  any  preparation  or  moderator  training  before  the  first  day  of  class.  A grad  student  that  I was  promised  to 
take  on  the  role  of  the  moderator  dropped  out  and  I was  left  with  trying  to  convince  a student  to  take  on  the  daunting 
task  of  moderating.  Mimi  Denomme  did  volunteer  but  a month  passed  before  the  training  could  take  place  due  to 
her  out  of  class  responsbilities.  She  had  three  small  children,  a full  time  job  and  was  taking  three  other  classes.  This 
gave  her  next  to  no  time  for  moderating.  I was  quite  frustrated  and  disappointed  with  the  amount  of  time  and  effort 
that  Mimi  devoted  to  moderating,  but  I certainly  understood  the  underlying  reasons  for  this  lack  of  participation. 

Stanley  Pike  was  not  sold  on  the  idea  of  interaction  being  a long  time  lecturer  who  easily  fell  into  the 
correspondence  course  model  of  distance  leaning.  He  neither  received  or  solicited  any  training  in  the  differences 
between  a face-to-face  course  and  one  conducted  at  a distance.  Stanley  felt  that  he  had  put  enough  time  into  the 
course  by  being  required  to  send  short  emails. 

It  was  a full  month  after  the  start  of  the  course  before  the  conferences  started,  and  two  weeks  after  that  that 
I convinced  Stanley  that  a participation  requirement  was  a critical  component  of  conferencing.  Hacker  and  Wignall 
(1997)  observed  that  participation  of  students  with  low  degrees  of  computer  experience  declined  significantly  over 
time.  Therefore,  a minimum  level  of  participation  should  be  mandated  at  the  start  of  the  conference.  This  can  be 
accomplished  either  by  establishing  a written  or  verbal  contract  between  the  moderator  or  teacher  and  the  students 
(Rohfeld  & Hiemstra,  1995).  Eastmond  and  Ziegahn  (1995)  maintained  that  along  with  mandating  a certain  number 
of  messages  per  week,  that  there  be  standards  established  for  the  quality  and  relevance  of  the  messages.  They 
suggested  that  participation  accounted  for  30%  of  the  course  grade.  It  has  also  been  proposed  that  the  amount  and 
quality  of  participation  should  be  able  to  raise  or  lower  assessment  by  a full  letter  grade  (Cifuentes,  Murphy,  Segur, 
& Kodali,  1997).  Many  conferences  take  time  to  get  established  and  spark  general  interest  to  the  extent  that  the 
members  need  no  incentive  to  post  messages;  therefore,  it  is  important  to  establish  a mandate  to  give  the  conference 
time  to  come  into  its  own. 

Results 

Mimi  tried  her  hand  at  welcome  messages  for  each  of  the  three-conferencing  spaces  within  her  conference 
(Firm  two).  I decided  that  three  spaces  made  sense  from  an  organizational  perspective  and  to  eliminate  confusion  on 
where  to  post.  What  was  developed  was  The  Coffee  Shop,  a social  space,  The  Board  Room,  a content  oriented  space 
and  The  Tech  Room,  a place  for  technical  problem  solving.  Each  of  the  other  two  groups  only  had  one  group  space. 

I gave  Mimi  feedback  on  my  feelings  on  what  would  work  and  provoke  discussion  and  what  wouldn't. 
Soon  we  agreed  upon  three  messages  that  would  suffice. 
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The  number  of  messages  posted  per  week  shed  some  light  on  the  scope  of  the  firms  and  serves  a point  of 
reference  and  departure. 

Figure  1.  Total  postings  per  week  in  each  firm 


Figure  one  shows  many  more  messages  being  posted  in  group  two  (the  group  with  a trained  moderator  than 
the  other  two  groups,  but  one  might  easily  see  a flaw  in  this  by  positing  that  the  majority  of  messages  were  written 
by  the  moderator. 

Figure  2.  Total  postings  per  week  in  each  firm  with  the  moderator  postings  removed . 


0 Firm  1 
■ Firm  2 
□ Firm  3 
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Subtracting  the  firm  2 moderator  messages  from  total  messages  still  shows  many  more  messages  posted  to 
firms  two  than  in  the  other  two  firms.  What  happened  in  the  firms  to  make  such  a difference? 

Firm  One 

At  the  start  of  data  collection,  firm  1 demonstrated  no  particular  leadership.  Messages  were  concerned  with 
organizational  issues  such  as  questions  dealing  with  due  dates  for  assignments  and  questions  regarding  grading 
procedures.  A number  of  people  were  confused  regarding  what  was  expected  of  them  and  how  to  find  Supreme 
Court  cases  required  for  homework.  There  was  one  thread  that  went  on  for  a number  of  weeks,  soliciting  employees 
for  a security  firm.  This  thread  upped  the  message  count  considerably.  Overall,  there  was  no  focus  to  this  firm  in 
the  first  few  weeks.  The  threads  were  mostly  concerned  with  asking  single  questions  and  receiving  single  answers. 
There  was  quite  a bit  of  talk  about  how  much  harder  the  course  was  than  what  was  expected,  which  also  became  a 
major  concern  in  other  firms. 

Within  a few  weeks  the  firm  evidenced  a strong  sense  of  pointlessness  of  participating  in  the  discussions 
whether  it  was  for  credit  or  not.  Since  the  firm  was  not  in  any  way  integrated  into  the  class,  the  majority  of 
participants  considered  it  a waste  of  time  and  by  the  end  of  the  course,  the  people  who  did  participate,  and  that 
number  dropped  every  week,  were  posting  messages  without  any  content  just  because  it  was  a requirement. 

DO  NOT  WASTE  YOUR  TIME  READING  THIS!!  I’m  just  sending  an  email  to  fulfill  the  three- 
a-week  requirement.  Unfortunately  I have  nothing  to  say  or  ask,  so  I am  writing  about  my  writing 
about  nothing. 

Soon  thereafter,  most  message  traffic  from  Firm  one  stopped  due  to  no  purpose,  no  leadership  and  no 
momentum. 

Community  was  not  built  in  firm  one  since  the  participants  never  found  a center  or  any  salient  reason  to  be 
there  at  all.  It  was  generally  considered  a waste  of  their  time. 

Firm  Three 

During  the  first  week  of  messaging  Karla  Schwartz  asserted  leadership  of  group.  Karla  had  a strong 
personality  and  quickly  fomented  discontent  with  the  amount  of  homework  that  the  class  was  responsible  to 
accomplish  each  week.  Although  the  group  had  no  indication  that  the  Instructor  was  not  reading  the  messages  Karla 
started  a thread  entitled  'Dr.  Pike  is  Insane',  making  the  point  that  the  workload  is  totally  unreasonable.  This  is  a part 
of  a representative  message  from  Karla: 

I am  sorry  but  I must  vent  again.  I am  absolutely  sick  of  this  sh*t.  I have  answered  18  questions 
and  I still  have  like  15  to  go.  This  is  absolutely  ridiculous  and  I am  SICK  OF  IT!  This  class  is 
taking  up  my  whole  goddamn  life  and  I do  nothing  but  work  and  come  home  and  work  on  this 
sh*t. 

This  created  a mob  mentality  where  everyone  seemed  to  jump  on  the  'bandwagon'  and  state  that  they  also 
weren't  going  to  take  it.  The  class  decided  to  distribute  answers  to  their  take  home  midterms  and  discussed  emailing 
each  other  long  homework  assignments  where  they  intended  to  change  the  wording  so  it  wouldn't  appear  as 
plagiarism.  These  messages  were  posted  in  public  forums  demonstrating  uninhibited  behavior  that  many  researchers 
have  noted  as  being  a inherent  in  asynchronous  communication. 

Eventually  the  workload  was  decreased  and  message  traffic  lessened  to  the  point  of  being  nearly  non* 
existent.  Since  there  was  nothing  left  to  complain  about,  they  had  nothing  to  say. 

Community  was  built  in  firm  three,  but  not  the  sort  proposed  by  advocates  of  online  learning.  What  built 
this  community  was  group  dissent  and  communal  complaints  lead  by  Karla  Schwartz  and  fomented  by  the  members 
of  the  firm  that  posted  messages.  During  the  week  of  the  midterm  eight  people  posted,  followed  by  seven  people 
just  before  Dr.  Pike  changed  the  assignments.  Once  the  change  was  made  only  four  people  posted.  Once  there  was 
nothing  left  to  rally  around,  everyone  seemingly  packed  up  and  went  home.  There  was  nothing  left  to  talk  about. 
Firm  Two 

The  amount  of  intervention  by  the  moderator  was  much  less  than  what  I,  at  first  considered  adequate,  the 
conference  started  quite  late  into  the  course  the  conference  was  generally  well  received  and  an  enjoyable  and  helpful 
addition  to  the  class.  An  interesting  side-note  is  that  in  interviewing  the  Instructor,  his  opinion  was  that  the 
conferencing  had  nothing  to  do  with  the  class,  most  of  the  attitude  surveys  from  members  of  Firms  two  stated  that 
the  messaging  was  well  integrated  into  the  class. 

Community  was  built  in  firm  two.  There  was  a heavy  social  use  of  The  Coffee  Shop  section  that  the  firm 
used  to  connect  and  form  relationships.  There  was  a large  amount  of  personal  information  revealed  as  an  ongoing 


system  of  empathy  and  warmth  was  developed.  What  built  the  community  was  the  construction  of  a warm  and 
friendly  atmosphere  prompted  by  Mimi,  and  the  camaraderie  that  grew  out  of  what  became  a place  to  come  for 
support  friendship  and  humor. 

Summary 

This  study  attempted  to  explore  the  effects  of  a trained  moderator  in  an  asynchronous  distance  learning 
class.  The  study  was  designed  to  use  grounded  theory  and  narrative  research  to  report  on  the  experiences  of  a 
graduate  assistant  expected  to  be  trained  as  a moderator,  and  the  members  of  the  computer  conference  managed  by 
the  moderator.  These  experiences  were  to  be  compared  with  the  experiences  of  two  other  segregated  computer 
conferences,  each  run  by  a moderator  that  would  change  each  week.  These  moderators  would  receive  no  formal 
training  of  any  sort.  The  computer  conferences  were  to  support  a distance  learning  course  offered  by  the  School  of 
Legal  Services  at  a campus  of  a large  southwestern  university. 

The  study  almost  did  not  take  place  due  to  a litany  of  problems,  including  the  graduate  assistant  meant  to 
undergo  the  moderator  training  dropping  out,  the  instructor  experiencing  life-threatening  medical  problems,  and  a 
course  that  was  previously  devoid  of  any  form  of  interaction  among  the  students.  The  instructor,  a long  time 
lecturer,  had  no  distance  learning  training  and  the  resulting  class  resembled  an  electronic  correspondence  course 
with  little  regard  given  to  student-student  interaction. 

A student  volunteered  to  undergo  training  and  moderate  one  of  three-segregated  computer  conferences  for 
the  semester.  Time  pressures  and  other  commitments  prevented  her  from  practicing  a number  of  the  techniques  of 
good  conference  moderation.  It  became  necessary  for  me  to  be  in  constant  contact  with  the  moderator,  heavily 
guiding  her  actions  until  late  into  the  semester.  The  other  two-segregated  computer  conferences  did  not  employ 
moderators  due  to  technical  complications. 

The  study  found  that  even  with  a low  degree  of  intervention  the  use  of  moderation  techniques  allowed  the 
moderated  group  to  form  a community  based  on  camaraderie,  support,  and  warmth.  Conference  messages  in  this 
group  amounted  to  over  2.5  times  that  of  the  next  highest  group.  It  was  also  found  that  structuring  places  to  post 
messages  by  dividing  the  conference  into  a content  oriented  space,  a social  space,  and  a technical  space  had  positive 
implications,  which  assisted  and  at  times  became  a substitute  for  certain  moderation  techniques. 

Analysis  of  the  other  two  groups  found  one  group  forming  no  community  whatsoever  and  the  other  group 
forming  a community  based  upon  group  dissent  and  communal  complaint  initiated  by  a strong  personality  and  what 
the  group  considered  an  inordinate  amount  of  work.  This  group  demonstrated  extremely  uninhibited  behavior  using 
words  and  concepts  that  would  be  inappropriate  in  a classroom  situation. 

In  a discussion  of  how  online  courses  are  constructed  on  the  campus  under  study,  it  was  discovered  that 
there  was  a wide  variety  of  overlapping  options  that  could  easily  become  confusing.  Major  emphasis  was  given  to 
putting  courses  online  while  little  thought  was  given  to  the  appropriateness  of  the  pedagogy  that  should  be 
employed. 
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Abstract 

The  purpose  of  this  research  study  was  to  determine  whether  a specific  information  problem-solving  skills 
model  was  an  effective  metacognitive  scaffold  for  students  solving  information-based  problems.  Specifically,  thirty- 
five  eighth  grade  students  in  two  intact  classes  were  asked  to  write  newspaper  articles  that  summarized  the  events 
surrounding  the  Selma  March  during  the  African-American  Civil  Rights  Movement.  Achievement  and  attitudinal 
data  were  collected  at  the  end  of  the  treatment  period,  and  observational  data  were  collected  throughout  the 
treatment  period.  One  class  of  students  followed  the  procedures  of  the  Eisenberg  and  Berkowitz  Information 

Problem  Solving  model  while  the  other  followed  the  procedures  provided  by  the  classroom  teacher. 

Results  revealed  that  the  students  following  a specific  metacognitive  scaffold  performed  better  on  the  achievement 
measure.  However,  the  students  using  a teacher-managed  process  reported  more  positive  attitudes  than  the 
scaffolded  students.  Possible  reasons  for  the  differences  in  reported  attitudes  between  the  two  groups  include 
differing  responsibility  levels  of  students  and  teachers  within  the  groups  as  well  as  differences  in  time  spent  in  on- 
task  behaviors  throughout  the  study. 

Introduction 

Information  literacy  has  been  a topic  of  research  for  a number  of  years.  Through  the  examination  of 
searcher  behavior,  several  models  have  been  developed  that  describe  the  processes  used  by  people  who  are  seeing 
information  (Eisenberg  & Berkowitz,  1990;  Kuhlthau,  1983;  Stripling  & Pitts,  1988).  Organizations  such  as  the 
American  Association  for  School  Librarians  and  the  American  Library  Association  have  collaborated  to  develop 
standards  for  information  literacy.  These  standards  call  for  students  who  access  information  efficiently  and 
effectively,  evaluate  information  critically  and  competently,  and  use  information  accurately  and  creatively 
(American  Association  of  School  Librarians,  1998).  Students  who  use  information  in  this  manner  to  explore  and 
solve  problems  are  identified  as  being  information  literate.  It  has  been  noted  that  students  need  various  resources, 
tools  and  scaffolds  to  support  their  efforts  to  solve  the  educational  problems  they  encounter  (Hannafin,  Land,  & 
Oliver,  1999;  Hannafin,  Hall,  Land,  & Hill,  1994).  The  role  that  scaffolding  plays  in  student  achievement  has  also 
been  explored  (Saye  & Brush,  1999;  Hannafin  et  al.,  1994;  Hannafin,  Hannafin,  Land,  & Oliver,  1997a).  This  study 
sought  to  enrich  the  body  of  knowledge  concerning  the  role  of  metacognition  and  metacognitive  scaffolds  in 
supporting  student  research  activities. 

Metacognition 

Metacognition  has  been  described  as  thinking  about  thinking.  More  specific  definitions  include  references 
to  knowledge  and  control  of  factors  that  affect  learning,  such  as  knowledge  of  self,  the  task  at  hand,  and  the 
strategies  to  be  employed  (Baker  & Brown,  1984;  Palincsar  & Brown,  1981).  In  order  to  perform  metacognitively 
learners  must  be  able  both  to  be  aware  of  their  own  cognitive  activities,  and  to  control  and  monitor  that  cognitive 
activity.  The  distinction  between  awareness  and  control  was  examined  in  order  to  determine  what  differences 
existed  between  learning  disabled  and  non-learning  disabled  students.  Slife,  Weiss,  and  Bell  (1985)  found  that  when 
these  two  groups  of  students  were  compared  in  metacognitive  skill  activities,  the  differences  were  in  metacognitive 
strengths  rather  than  skills  deficiencies. 

McGregor  (1993)  examined  the  thinking  processes  that  students  engaged  while  writing  research  papers. 
She  found  that  students  seemed  to  be  unaware  of  their  own  cognitive  processes.  That  is,  “students  do  not 
instinctively  operate  in  a metacognitive  manner”  (McGregor,  1993  p.  131).  Other  researchers  have  found  that 
student  success  in  a learning  environment  is  impacted  by  the  lack  of  metacognitive  ability  of  the  students  (Hill, 
1995;  Land  & Hannafin,  1997).  This  lack  of  metacognitive  skill  and  awareness  supports  the  need  for  instruction 
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that  teaches  learners  to  “plan,  implement  and  evaluate”  (Palincsar,  1986  p.  123)  strategies  for  learning  and  problem- 
solving. 

Scaffolding 

How  can  teachers  provide  the  instruction  students  need  in  order  to  develop  strong  metacognitive  skills  that 
are  inherently  difficult  both  to  observe  and  teach?  One  way  is  to  incorporate  the  use  of  scaffolds  into  the 
curriculum.  While  the  basic  concept  of  scaffolding  has  been  defined  as  a support  structure  for  learners  engaged  in 
activities  just  beyond  their  independent  abilities  (Vygotsky,  1978),  some  have  further  delineated  differences  between 
specific  types  of  scaffolding.  Hannafin  et  al.  (1999)  identified  four  different  types  of  scaffolds,  metacognitive, 
procedural,  conceptual,  and  strategic.  Figure  1 provides  an  overview  of  these  scaffold  types  and  the  situations 
where  they  might  be  used. 


Scaffold  Type 

Description 

Used  when... 

Metacognitive 

Guidance  in  what  to  think  during  a 
learning  activity 

Students  are  engaged  in  an 
independent  metacognitive  activity 
such  as  research-based  problem 
solving 

Procedural 

Assistance  with  a particular  tool  or 
feature  of  a learning  environment 

Technology  is  being  utilized;  job 
aids  are  needed 

Conceptual 

Assistance  with  what  to  consider; 
Vygotskian  scaffolding 

The  various  possible  methods  for 
achieving  success  can  be  known 
ahead  of  time  by  the  teacher 

Strategic 

Guidance  in  the  approach  that 
might  be  needed  in  a learning 
situation 

Alternative  strategies  have  not  been 
considered  by  students; 
participation  in  planning  and 
implementing  decision  making 
skills  in  open-ended  learning 
environments. 

Figure  1.  Scaffold  descriptions  and  uses 

Research  indicates  that  how  scaffolds  are  used  in  various  learning  situations  has  impacted  student 
achievement  and  attitudes  (Saye  & Brush,  1999;  Hill,  1995;  Krajcik,  Soloway,  Blumenfeld,  & Marx,  1998).  For 
example,  Oliver  (1996)  and  Brush  and  Saye  (2000)  both  found  that  the  use  of  a scaffold  rather  than  its  presence 
impacted  student  success  within  a particular  learning  situation.  Other  researchers  (Hill,  1995;  Land  & Hannafin, 
1997)  suggest  that  deficiencies  found  in  student  metacognitive  skill  could  be  mitigated  through  the  use  of  strong 
metacognitive  scaffolds.  In  addition,  the  school  library  community  has  recognized  the  need  for  students  to  posses 
strong  metacognitive  skills. 

Information  Problem-Solving 

Within  the  school  library  media  community  several  researchers  have  studied  searcher  behavior  in  a variety 
of  contexts,  including  print  (Dreher,  1993;  Dreher  & Sammons,  1994),  Electronic  (Marchionini,  1989),  and 
multimedia  (Perzylo  & Oliver,  1992)  environments.  Others  sought  to  describe  the  search  process  in  descriptive 
(Kuhlthau,  1993),  and  prescriptive  (Eisenberg  & Berkowitz,  1990;  Stripling  & Pitts,  1988)  ways. 

A common  theme  through  the  research  on  information  seeking  involves  the  need  to  increase  the 
metacognitive  skills  of  students.  A general  metacognitive  scaffold  is  needed  so  that  students  do  not  have  to  rely  on 
situation  specific  scaffolds  each  time  they  encounter  a problem  or  unfamiliar  situation  (Costa,  1984).  Several 
information  problem-solving  models  exist  (Kuhlthau,  1983;  Stripling  & Pitts,  1988;  Eisenberg  & Berkowitz,  1988) 
that  could  function  as  metacognitive  scaffolds.  The  Eisenberg  and  Berkowitz  Information  Problem-Solving  (IPS) 
model  is  well  suited  for  use  in  this  capacity.  Figure  2 provides  an  illustration  of  the  IPS  model.  Each  step  or  task  is 
comprised  of  two  sub-tasks  that  students  should  accomplish  in  order  to  use  information  in  an  effective  and  efficient 
manner  to  solve  educational  problems. 
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Figure  2 . The  Big  Six  Information  Skills  (Eisenberg  & Berkowitz,  1990) 


Integrating  Scaffolding,  Metacognition  and  IPS 

The  implementation  of  an  information  problem-solving  model  involves  more  than  teaching  students  a 
series  of  steps  and  directing  them  to  the  research  materials  in  the  library.  In  order  for  learning  to  occur  in  these 
situations,  students  must  actively  interact  with  materials  and  information  in  such  a way  as  to  construct  their  own 
meaning  from  the  interaction  (Kuhlthau,  1993).  However,  as  has  been  previously  noted,  this  does  not  always  occur 
particularly  with  student  who  are  unfamiliar  with  the  research  process  (McGregor,  1993;  Perkins,  1991;  Steinberg, 
1977,  1989).  However,  if  the  “purpose  of  strategy  instruction  is  to  influence  how  the  learner  interacts  with  the 
learning  situation”  (Palincsar,  1986  p.  118),  then  the  use  of  IPS  as  a metacognitive  scaffold  is  warranted. 

There  are  many  references  to  the  Eisenberg-Berkowitz  IPS  model  on  the  World  Wide  Web  and  in  the 
literature  read  by  professional  school  library  media  specialists.  While  this  body  of  anecdotal  evidence  is  compelling 
as  to  the  far-reaching  effects  this  model  has  had  on  educational  practices,  it  is  lacking  in  rigorous  research  to  support 
the  conclusions  presented.  The  only  research  that  discusses  the  impact  this  IPS  model  might  have  on  student 
achievement  was  conducted  by  the  authors  of  the  model  (Eisenberg,  1999;  Eisenberg  & Berkowitz,  1998).  This 
case  study  reported  an  improvement  from  53%  to  95%  of  students  passing  a high  school  American  History 
examination  over  a period  of  one  year. 

While  a variety  of  research  has  been  conducted  in  order  to  understand  the  information  seeking  behaviors  of 
searchers  (Kuhlthau,  1991;  Marchionini,  1989;  Stripling  & Pitts,  1988)  and  there  has  been  an  identified  need  to 
strengthen  metacognitive  skills  in  students  (Hill  & Hannafin,  1997;  Brush  & Saye,  in  press;  Oliver  & Perzylo,  1994) 
a detailed  examination  of  the  effectiveness  of  particular  information  problem-solving  models  has  not  been 
conducted.  If  a particular  model  were  shown  to  be  effective  in  strengthening  metacognitive  skills  in  students  several 
of  the  performance  gaps  identified  in  research  could  be  addressed.  Students  would  effectively  and  efficiently  access 
and  use  information,  students  would  monitor  their  own  thought  processes,  teachers  would  design  effective 
scaffolded  problem  spaces,  and  students  would  begin  to  transfer  problem-solving  skills  from  one  academic  situation 
to  another. 

Method 

Thirty-five  students  in  two  eighth-grade  social  studies  classes  in  a major  southwestern  city  participated  in 
the  study,  divided  equally  between  male  and  female  genders.  The  researcher  acted  as  both  participant  and  observer 
during  the  course  of  the  study  by  conducting  one  of  the  classes  for  the  students  and  providing  technical  support  for 
the  participating  teacher  while  he  conducted  his  class.  The  participating  teacher  conducted  the  other  class  for  the 
study  according  to  his  established  classroom  procedures.  Students  in  this  study  were  provided  with  their  own 
computer  (either  a laptop  or  a desktop  system)  to  use  for  research  activities. 

A two  group  (scaffolded  vs.  non-scaffolded),  quasi-experi mental  design  was  implemented  to  determine 
achievement  differences  within  this  study.  One  group  (scaffolded  class)  received  training  and  guidance  in  the  use  of 
a specific  IPS  model  while  completing  their  reports  and  the  other  group  (non-scaffolded  class)  received  no  explicit 
IPS  training.  Each  student  completed  a 15  item,  multiple  choice  pre-test  prior  to  the  onset  of  the  study.  Items 
covered  knowledge-level  information  about  the  Selma  March  as  well  as  the  African-American  Civil  Rights 
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Movement.  Subject  matter  content  was  delivered  via  Decision  Point!  (DP),  an  “integrated  set  of  multimedia  content 
resources  and  tools”  (Saye  & Brush,  1999,  p.  11)  relating  to  the  African-American  Civil  Rights  Movement. 
Restricting  the  activity  to  a single  event  allowed  the  researcher  to  control  the  problem  space  students  engaged  while 
allowing  students  relative  freedom  to  explore  an  even  however  they  deemed  appropriate  (Oliver  & Perzylo,  1994; 
Saye  & Brush,  1999;  Yang,  1997).  The  researcher  created  job-aids  for  all  participating  students  to  use  during  the 
study  activities.  These  procedural  scaffolds  (Hannafin  et  al.,  1999)  helped  students  maintain  focus  and  remain 
oriented  in  the  open-ended  environment. 

Students  in  the  scaffolded  class  were  provided  with  an  additional  job-aid  to  remind  them  of  the  steps 
involved  in  the  information  problem-solving  process  (see  figure  1).  Some  elements  of  this  metacognitive  scaffold 
were  incorporated  directly  into  DP  through  the  “guides”  and  “journal”  tabs  of  the  electronic  notebook.  While  these 
scaffolds  were  available  to  all  of  the  students,  only  the  students  in  the  scaffolded  class  were  explicitly  told  to  use 
them.  “Guides”  questions  consisted  of  a series  of  questions  organized  around  the  five  questioning  words  associated 
with  news  article  writing  (who,  what,  where,  when,  why).  The  journal  contained  prompts  for  the  students  to 
complete  that  provided  guidance  in  thinking  about  their  own  progress,  and  in  making  a plan  for  the  next  class 
meeting  day. 

The  study  was  conducted  over  1 1 class  days,  with  one  additional  class  day  being  used  for  the 
administration  of  the  study  pre-test.  The  participating  classes  met  on  alternating  days,  beginning  each  Monday. 
Classes  that  met  on  Monday  or  Wednesday  lasted  85  minutes,  while  classes  that  met  on  Friday  lasted  35  minutes. 
Prior  to  the  beginning  of  the  study  one  of  the  classes  was  chosen  to  receive  the  scaffolded  activities.  The  other  class 
received  instruction  that  the  classroom  teacher  designed  and  felt  was  appropriate  for  the  unit  as  a whole. 

One  week  prior  to  the  beginning  of  the  study  activities  the  researcher  visited  the  classroom  in  order  to  acclimate 
herself  to  the  regular  activities  for  each  class.  During  this  week  the  researcher  administered  the  pre-test  to  the 
students.  Although  the  scaffolded  class  scored  slightly  higher  (M=4.5)  than  the  non-scaffolded  class  (M=4.0)  a 
Mann-Whitney  analysis  of  the  pretest  scores  indicated  that  there  were  no  differences  in  prior  knowledge  between  the 
two  groups,  U(  18,17)  = 1 16.50,  p = .36.  Upon  completion  of  the  pretest,  an  introductory  activity  was  conducted 
with  both  classes.  The  activity  consisted  of  a short  scavenger  hunt  using  the  DP  software.  This  helped  insure  that 
students  were  familiar  with  the  basic  components  of  the  event  that  they  investigated  as  a part  of  their  final  activity. 
After  completion  of  the  scavenger  hunt,  the  structured  unit  activities  began. 

Each  class  received  a different  first  activity  to  begin  the  study.  The  first  activity  for  the  scaffolded  class 
was  an  information  problem-solving  training  session.  The  orientation  focused  on  the  IPS  process  students  were 
asked  to  engage  during  the  study  activity.  The  first  activity  for  the  non-scaffolded  class  was  an  introduction  to 
newspaper  article  writing,  conducted  by  the  teacher.  Once  each  class  completed  their  orientation  activities,  the 
students  spent  three  class  days  collecting  information  related  to  the  Selma  March  and  creating  their  initial  (rough- 
draft)  reports.  They  were  given  two  class  periods  to  make  final  revisions  to  their  work.  Students  in  the  scaffolded 
class  determined  their  own  methods  of  information  gathering  with  only  mild  guidance  from  the  researcher. 

Students  in  the  scaffolded  class  began  their  study  activities  by  determining  exactly  what  was  being  asked  of 
them.  They  then  generated  lists  of  questions  that  they  could  use  to  answer  and  fulfill  their  article  requirements. 
Once  the  questions  were  generated  students  sued  the  DP  database  to  find  answers  to  their  questions.  They  also  used 
the  questions  in  the  Guides  section  of  the  DP  notebook  to  focus  their  research.  Once  the  students  finished  answering 
their  own  questions  and  those  featured  in  the  guides  questions,  they  used  that  information  to  create  a handwritten 
rough  draft.  Students  then  used  the  scoring  guide  to  evaluate  their  neighbor’s  article.  Students  took  those  comments 
and  made  revisions  to  their  work  and  submitted  the  final  form  of  their  article.  The  teacher  guided  his  students 
during  the  newspaper  article  writing  process  according  to  established  classroom  procedures.  After  the  initial 
introduction  to  the  structure  of  news  articles  the  classroom  teacher  told  his  students  to  “find  the  information  you 
need  in  order  to  write  your  articles.”  Students  turned  in  handwritten  rough  drafts  that  the  teacher  took  home  to  edit. 
The  next  class  period,  he  returned  the  rough  drafts  to  the  students  so  that  they  could  make  the  indicated  changes. 
Students  turned  in  their  final  copies  with  an  attached  picture  on  a separate  piece  of  paper. 

The  articles  that  the  students  created  were  news  articles  that  reported  on  the  events  surrounding  the  Selma 
March.  Students  in  both  classes  submitted  the  reports  in  written  format.  As  a part  of  the  IPS  scaffold,  students  in 
the  scaffolded  class  were  given  the  criteria  for  grading  prior  to  their  submission  of  the  assignment.  The  reports 
covered  the  following  elements  of  the  Selma  March:  a general  overview  or  timeline  of  events,  key  people  involved 
in  the  event,  causes  of  the  event,  and  results  of  the  event.  Observers  took  field  notes  and  audiotaped  each  class 
session  in  order  to  collect  qualitative  data  concerning  student  engagement,  attitudes,  and  behaviors  during  the 
information  problem-solving  process.  After  each  of  the  classes  the  teacher  and  researcher  conferred  for  a short 
debriefing  session.  During  this  time  the  teacher  had  the  opportunity  to  discuss  any  significant  occurrences  that  the 
researcher  may  have  missed  during  the  class  session  and  share  his  impressions  of  student  attitudes  and  progress. 
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All  of  the  students  participating  in  the  study  completed  exit  surveys  during  the  final  class  period.  The  4- 
point  Likert-type  survey  collected  attitudinal  data  from  the  students,  such  as  the  students’  feelings  about  the  unit,  the 
topic,  their  preferred  way  of  learning  social  studies  information  and  their  feelings  about  future  research  projects.  In 
addition,  students  were  asked  to  respond  to  open-ended  questions  regarding  the  elements  of  the  project  they  liked 
and  disliked  as  well  as  whether  they  would  recommend  the  project  to  their  peers.  Following  the  submission  of 
student  reports  the  researcher  conducted  exit  interviews  with  selected  students.  The  classroom  teacher 
recommended  students  for  selection  based  on  their  willingness  to  be  interviewed  and  their  ability  to  express  and 
elaborate  on  their  thoughts.  The  interviews  were  conducted  in  pairs  to  help  alleviate  any  anxiety  associated  with  the 
interview  process  as  well  as  to  gain  a more  complete  understanding  of  what  the  student  knew  regarding  the  research 
study  (Graue  & Walsh,  1998).  Interview  questions  were  based  on  journal  entries  and  observations  from  class 
sessions.  Students  were  asked  to  explain  and  expand  on  comments  made  in  their  journals,  to  verify  assertions  made 
during  classroom  observations,  and  to  give  their  opinions  about  the  study  activities. 

Two  neutral  scorers  used  the  evaluation  rubric  to  assign  scores  to  each  report.  To  help  ensure  inter-rater 
reliability  each  scorer  independently  scored  each  report  and  then  compared  their  ratings.  If  the  scores  were  highly 
divergent  the  researcher  conferred  with  the  scorers  to  clarify  any  confusion  about  the  rubric,  and  how  to  interpret 
student  reports  so  that  subsequent  grading  would  have  a higher  level  of  agreement  between  the  two  scorers.  At  the 
conclusion  of  this  procedure  the  correlation  between  the  two  groups  of  scores  was  .91 . 

Results 

Both  quantitative  and  qualitative  data  were  collected  during  this  study.  Results  as  they  pertain  to  data  types 
are  discussed  below. 

Newspaper  Articles 

A Mann-Whitney  test  of  the  report  scores  revealed  that  there  were  significant  differences  between  the 
report  scores  for  students  in  the  scaffolded  (N  = 18,  M = 12.72,  SD  = 1.64)  and  the  non-scaffolded  (N  = 15,  M = 

1 1 .00,  SD  = 1 .36)  classes,  U(  1 8, 1 5)  = 6 1 .00,  p<.0 1 . 

Student  Questionnaires 

Table  1 provides  an  illustration  of  student  responses  to  selected  questions  from  the  attitude  survey  (Nnon. 
scafToided=17,  NscafToided=  1 8).  Responses  were  provided  on  a four  point  Likert-type  scale  and  then  coded  as  either  agree 
or  disagree  for  analysis  purposes.  Chi-square  analyses  were  conducted  on  each  item  to  determine  differences  in 
attitudes.  There  were  significant  differences  in  the  attitudes  of  the  two  classes  regarding  items  “I  felt  smart  while 
doing  the  project,”  x2(l  ,N=35)=6.89,  p<.01  and  “I  would  like  to  do  more  projects  like  this  on  other  social  studies 
topics”  x20,N=35)=5.93,  p<.05.  There  was  also  consensus  on  several  other  survey  items.  The  majority  of  students 
in  both  classes  disagreed  with  the  statements:  “This  project  made  me  feel  nervous”  and  “This  project  made  me  feel 
dumb.”  Also,  the  majority  of  students  in  both  classes  agreed  with  the  statements:  “This  project  helped  me 

understand  The  Civil  Rights  Movement  better  than  if  I had  just  read  about  it  in  my  textbook,”  “I  felt  comfortable 
researching  topics  I know  little  about,”  and  “I  felt  comfortable  writing  my  newspaper  article  about  the  Selma 
March.” 

Open-ended  questions  asked  students  to  provide  for  the  researcher  things  they  liked  best  and  liked  least 
about  doing  the  project;  ways  to  improve  the  assignment;  things  they  would  have  liked  to  have  more  time  to  do;  and 
whether  they  would  recommend  this  project  to  other  eighth  grade  students.  Students  from  both  classes  liked  the 
computers  and  the  software  that  they  used  for  their  activities.  The  next  most  popular  answer  was  that  they  liked 
learning  about  the  Civil  Rights  Movement.  These  answers  included  responses  referring  to  the  era,  the  decade,  or  the 
event.  Other  comments  included  references  to  the  lack  of  homework  during  the  unit  activities,  the  format  of  the 
product  they  produced  and  being  able  to  do  something  different,  or  at  their  own  pace.  However,  the  scaffolded  class 
mentioned  the  research  process  in  greater  numbers  than  did  the  non-scaffolded  class. 

When  asked  what  they  liked  least  about  the  unit,  the  most  common  answer  from  both  classes  concerned  the 
physical  act  of  writing  the  paper.  Students  were  unable  to  use  a word  processor  to  write  their  papers  because  14  of 
the  18  students  in  each  class  were  using  laptop  computers  that  wee  not  connected  to  the  single  classroom  printer. 
There  were  also  two  sets  of  students  who  did  not  like  aspects  of  the  technology  used  and  aspects  of  having 
observations  conducted  in  their  classrooms.  Most  students  in  both  groups  responded  that  they  would  recommend 
this  project  as  well  as  similar  ones  to  other  eighth  grade  students.  Their  reasons  ranged  from  “because  it  was  fun 
and  interesting”  to  “it  is  an  important  event  for  people  to  learn  about.” 
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Table 1 Student  Attitudes. 


Class 

Scaffolded 

Non  Scaffolded 

Statement 

Agree 

Disagree 

Agree 

Disagree 

X2 

This  project  made  me  feel  nervous. 

11.1%  (2) 

88.9%  (16) 

11.8%  (2) 

88.2%  (15) 

.00 

I felt  smart  while  doing  this  project. 

38.9%  (7) 

61.1%  (11) 

82.4%  (14) 

17.6%  (3) 

6.89** 

This  project  helped  me  understand  The 
Civil  Rights  Movement  better  than  if  I had 
just  read  about  it  in  my  textbook. 

77.8%  (14) 

22.2%  (4) 

88.2%  (15) 

11.8%  (2) 

.67 

I would  like  to  do  more  projects  like  this 
on  other  topics  in  social  studies. 

50%  (9) 

50%  (9) 

88.2%  (15) 

11.8%  (2) 

5.93* 

I felt  comfortable  researching  topics  that  I 
know  little  about. 

83.3%  (15) 

16.7%  (3) 

82.4%  (14) 

17.6%  (3) 

.01 

I felt  comfortable  writing  my  newspaper 
article  about  the  Selma  March. 

76.5%  (13) 

23.5%  (4) 

82.4%  (14) 

17.6%  (3) 

.18 

This  project  made  me  feel  dumb. 

11.1%  (2) 

88.9%  (16) 

11.8%  (2) 

88.2%  (15) 

.00 

*p_<  .05;  **p_<  .01 


Classroom  Observations 

During  periods  of  direct  instruction,  students  in  both  classes  demonstrated  their  understanding  of  the  proper 
behavior  of  a school  classroom.  They  raised  their  hands,  tended  not  to  speak  all  at  once,  and  listened  to  the 
comments  of  their  classmates.  Also,  when  they  knew  the  answer  to  a question  from  a classmate  they  were  fairly 
quick  to  answer  and  provide  help. 

During  periods  devoted  to  research  activities  students  in  the  scaffolded  class  exhibited  behaviors  that 
suggested  they  were  more  self-directed  in  their  activities.  Students  in  the  non-scaffolded  class,  however,  spent  the 
majority  of  their  writing  time  traveling  back  and  forth  between  their  desks  and  wherever  the  teacher  was  standing. 
They  were  less  willing  to  make  decisions  on  their  own  and  relied  on  the  teacher  to  do  the  majority  of  their  editing. 

Students  in  both  classes  asked  many  questions  that  were  focused  on  the  technology  of  the  DP  database. 
These  questions  ranged  from  “Why  don’t  my  movies  plan?”  to  “How  do  I get  to  do  the  typing  party?”  Any 
questions  of  this  type  were  answered  by  the  researcher,  regardless  of  which  class  was  in  the  room.  Most  of  the 
comments  made  by  students  in  the  non-scaffolded  class  were  directed  at  the  teacher.  Usually,  a student  would  ask 
the  teacher  to  review  a sentence  or  paragraph  that  had  just  been  written  and  wait  for  specific  feedback  from  him. 
Comments  between  students  generally  were  initiated  only  when  the  teacher  was  extremely  busy,  and  were 
superficial  in  nature.  For  example,  a common  question  students  would  ask  each  other  was  “how  do  you  spell...” 
Also,  they  would  ask  their  neighbor  how  to  find  a particular  video  or  picture  that  was  displayed  on  their  computers 
or  ask  for  help  with  technical  issues  on  the  CD-ROM.  The  majority  of  the  comments  directed  to  the  teacher  had  to 
do  with  the  physical  construction  of  their  final  products.  Students  asked  the  teacher  to  “check-over”  each  sentence 
as  they  wrote  them.  Sometimes,  students  would  ask  the  teacher  to  hold  their  papers  and  call  them  when  he  had 
finished  with  the  ones  ahead  of  them  in  line.  Students  who  used  his  technique  for  holding  their  place  in  line  were 
observed  talking  about  movies,  parties,  homework  assignments  from  other  classes,  and  other  topics  that  had  nothing 
to  do  with  the  Selma  March.  Once  the  students  reached  the  teacher  the  most  frequently  asked  question  was,  “Is  this 
ok  for  a lead/title/sentence?” 

Students  in  the  scaffolded  class  also  asked  each  other  spelling  and  technical  questions,  but  they  interacted 
with  each  other  in  terms  of  the  content  they  were  exploring.  One  student  was  heard  to  ask,  “Who  was  this  Lyndon 
Johnson?”  The  reply  was,  “Duuuhhh,  he  was  the  President  during  all  of  this  stuff! ” Students  sitting  next  to  each 
other  were  observed  talking  about  the  information  they  found  in  the  articles  they  read  during  the  activity.  Usually 
this  was  in  connection  with  a question  they  asked  the  researcher  about  specific  information  they  did  not  understand, 
or  had  difficulty  finding  within  the  CD-ROM.  Students  also  discussed  the  social  issues  they  were  finding  in  the 
videos  and  articles  of  the  DP  database.  Two  students  were  heard  discussing  the  video  of  the  attack  on  the  marchers 
at  the  Edmund  Pettus  bridge.  They  discussed  how  the  video  demonstrated  elements  of  racism  based  on  the  attacking 
dogs  and  police  officers  only  “going  after”  the  African-American  people. 

During  the  final  portion  of  the  activity  the  students  in  the  scaffolded  group  were  asked  to  review  their 
neighbor’s  reports.  Comments  during  this  activity  were  centered  around  the  structure  of  the  article  itself.  Several 
students  were  heard  to  say  “the  green  guide  [scoring  guide]  says  that  you  have  to  have  six  paragraphs,  and  you  only 
have  one,”  or  similar  types  of  comments.  No  comments  were  made  about  spelling  or  grammar,  only  about  the 
physical  layout  of  the  articles. 
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During  the  two  class  periods  designated  as  primarily  research  days  the  noise  level  in  the  class  was 
markedly  lower  than  during  any  other  days  during  the  study.  Students  in  both  classes  were  observed  intently 
viewing  their  computer  screens.  The  students  in  the  non-scaffolded  class  were  also  using  notebook  paper  and  pens 
to  take  notes.  However,  these  students  were  observed  copying  directly  (verbatim)  from  the  articles  onto  their 
papers.  For  the  most  part,  students  worked  independently.  Brief  comments  were  made  between  table  partners,  but 
the  research  process  was  primarily  conducted  on  an  individual  basis.  Students  in  the  scaffolded  class  were  typing 
directly  into  the  computer.  Upon  observation,  these  students  were  typing  the  answers  to  the  Guides  questions  that 
were  incorporated  into  the  DP  database.  Usually,  their  answers  were  typed  in  a list  format.  For  example,  one  of  the 
Guides  questions  was,  “who  was  involved  in  this  event?'’  A typical  student’s  entry  would  be,  “MLK,  black  voters, 
federal  judge,  Jimmie  Lee  Jackson,  President  Johnson.”  Some  students  were  seen  to  be  working  together  to  gather 
the  information  for  their  Guides. 

Student  behavior  on  the  days  designated  for  writing  rough  drafts  was  different  between  the  two  classes. 
The  students  in  the  non-scaffolded  class  alternated  their  activities  between  writing  one  or  two  sentences  at  a time 
and  waiting  for  the  teacher  to  check  their  work.  As  a result,  there  were  many  students  walking  around  the  room 
during  this  period  of  time.  During  the  consultations  with  the  teacher  most  students  in  the  non-scaffolded  class 
would  get  part  of  a question  out  and  then  the  teacher  would  interrupt  and  finish  it,  or  provide  the  answer  to  their 
question  without  letting  them  finish.  Many  times,  the  teacher  would  write  directly  on  the  students’  papers  with  the 
wording  that  he  wanted  them  to  use  and  they  would  copy  it  verbatim  into  the  next  revision  of  their  article.  In  most 
cases  the  teacher  was  in  possession  of  the  student’s  paper  and  pencil  during  these  consultations. 

These  behaviors  contrast  with  those  observed  during  the  scaffolded  class.  During  the  days  designated  as 
writing  days,  these  students  were  observed  going  back  and  forth  between  the  computer  and  their  papers.  While  they 
did  consult  with  the  researcher,  she  maintained  a different  relationship  with  the  students  than  observed  in  the  non- 
scaffolded  class.  When  students  asked,  “Is  this  an  ok  title?”  or  “Is  this  all  right  so  far?”  the  researcher  replied,  “You 
have  a copy  of  the  exact  scoring  guide  that  I will  use  to  give  you  a grade  on  the  article.  You  can  make  those 
decisions  yourself.”  After  answering  approximately  half  of  the  class’  questions  like  this,  the  types  of  questions 
shifted  to  content-oriented  or  technologically-focused  ones.  During  the  final  evaluation  activity  for  the  scaffolded 
class,  the  noise  level  increased,  but  upon  observation  this  was  due  to  increased  occurrences  of  debate  between 
students.  During  this  activity  students  were  more  likely  to  consult  with  the  researcher  to  clarify  the  elements  of  the 
scoring  guide.  After  the  peer  evaluation,  students  made  any  corrections  to  their  work  that  they  felt  were  necessary. 

Interviews  with  the  teacher  revealed  that  he  thought  the  unit  was  generally  a success  for  both  classes.  He 
said,  “I  thought  they  were  interested  and  engaged.”  He  also  noted  some  differences  between  the  two  classes  such  as 
attitudes  of  specific  students,  time  spent  in  on-task  behaviors,  and  the  quality  of  the  work  submitted  by  the  students. 

Two  of  the  students  in  the  researcher’s  group  were  seen  by  the  teacher  to  be  exceptionally  interested  and 
engaged  with  the  material.  One  of  these  students  exhibited  what  to  the  teacher  was  unusual  maturity  during  the 
completion  of  the  project.  Usually  this  particular  student  failed  to  work  up  to  his  potential  and  maturity  level,  yet 
still  managed  to  get  fairly  good  grades  in  school.  This  student’s  attitude  stood  out  to  the  teacher  because  it  was  so 
different  from  his  normal  behavior.  Other  students  demonstrated  differences  in  their  behavior  as  well.  One  student 
voiced  her  concern  over  the  quality  and  correctness  of  her  work  to  the  classroom  teacher.  “Her  paper’s  decent;  it’s 
not  great,  but  she  normally,  if  she  doesn’t  understand,  her  defense  mechanism  is  to  laugh  it  off  and  screw 
around... But  she  acted  like  she  wanted  to  do  this  or  was  concerned  about  am  I doing  this  right?  Is  this  good?”  A 
difference  the  teacher  noticed  between  the  two  groups  was  their  time  spent  on-task  during  the  research  assignment. 
“Your  group  was  a little  more  on  task  as  a whole  than  mine,”  said  the  teacher.  He  attributed  this  difference  to  the 
difference  in  presentation  styles  between  himself  and  the  researcher. 

The  teacher  also  thought  that  the  quality  of  the  student  reports  was  different  between  the  two  groups.  Prior 
to  evaluation  of  the  final  reports,  the  researcher  asked  the  teacher  to  predict  whether  or  not  there  would  be 
differences  in  the  scores  between  the  two  classes.  The  teacher  predicted  that  the  quality  of  the  students  in  the 
scaffolded  class  would  be  higher  than  in  the  non-scaffolded  class.  He  felt  that  the  steps  of  the  information  problem- 
solving model  provided  a structure  that  was  detailed  in  such  as  way  as  to  make  failure  difficult  to  achieve. 

Overall,  the  students  interviewed  from  both  classes  felt  that  their  experience  with  the  unit  was  a positive 
one  and  that  they  would  do  something  like  this  again,  that  the  teacher  should  have  provided  an  overview  of  the 
Selma  March,  and  that  they  wished  the  content  of  the  news  articles  were  organized  chronologically  rather  than  by 
information  type.  Differences  in  attitudes  between  the  two  classes  were  expressed  through  their  perceptions  of  the 
newspaper  article  assignment,  what  made  the  unit  fun,  the  role  of  the  teacher,  and  the  types  of  information  within 
the  DP  database.  The  students  in  the  non-scaffolded  class  felt  that  writing  newspaper  articles  was  a good  way  to 
process  information  and  demonstrate  their  understanding  of  a topic,  but  did  not  like  to  do  it.  When  the  researcher 
asked  students  from  the  non-scaffolded  class  how  they  would  improve  the  unit,  they  focused  on  providing  their 
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teacher  with  an  assistant.  They  felt  that  the  teacher  was  overextended  during  the  study  activities.  They  also  spoke 
strongly  about  having  to  wait  for  long  periods  of  time  to  get  their  questions  answered  about  their  articles.  This 
paralleled  the  responses  on  the  exist  surveys  where  one  student  stated  “It  would  have  been  better  if  [the  teacher]  had 
time  to  answer  everyone’s  questions.” 

Students  in  the  scaffolded  class  mentioned  different  aspects  of  the  unit  during  their  interviews.  Their 
comments  tended  to  focus  on  the  research  process  and  differences  in  the  types  of  information  they  encountered 
while  conducting  research  for  their  newspaper  articles.  These  students  indicated  that  the  implementation  of  the  unit 
was  cumbersome  to  them.  They  would  have  preferred  to  have  a single  sheet  of  paper  to  follow  rather  than  the 
packet  that  was  provided  for  them.  They  also  indicated  that  the  steps  [of  the  IPS  model]  were  generic  enough  that 
they  would  try  to  use  them  in  other  classes  for  other  research  projects.  Students  from  the  scaffolded  class  also  noted 
that  the  nature  of  the  information  they  explored  was  different  than  the  information  they  usually  used  during  social 
studies  classes.  These  students  differentiated  between  the  types,  and  value,  of  information  found  in  primary  sources 
and  in  textbooks.  They  did  this  despite  not  having  had  class  discussions  about  differences  between  primary  and 
secondary  sources.  For  these  students,  the  primary  source  documents  provided  more  authentic  and  accurate 
information  than  their  textbooks. 

Discussion 

Using  specific  information  problem-solving  models  as  metacognitive  scaffolds  has  not  been  widely  studied. 
This  research  suggests  that  a particular  information  problem-solving  model  might  act  as  an  effective  metacognitive 
scaffold  for  students  engaging  in  complex  research-based  activities. 

Results  showed  that  students  who  completed  research  writing  activities  (newspaper  article  creation) 
supported  by  the  IPS  model  created  newspaper  articles  that  were  more  accurate,  utilized  a wider  variety  of 
information  resources,  and  contained  richer  details  than  students  who  did  not  have  this  support.  Achievement  scores 
between  the  two  classes  of  students  differed  significantly,  with  the  mean  achievement  score  for  the  scaffolded  class 
being  nearly  two  raw  score  points  higher  than  the  non-scaffolded  class.  In  addition,  this  study  showed  that  the  use 
of  an  information  problem-solving  model  increased  student  engagement,  but  may  have  affected  the  attitudes  of 
students. 

There  are  two  potential  reasons  for  the  significant  differences  found  in  student  achievement  scores: 
increased  metacognitive  activity  and  differences  in  time-on-task  between  students  in  the  scaffolded  class  and 
students  in  the  non-scaffolded  class.  Metacognitive  skills  such  as  task  analysis,  strategy  selection,  and  self- 
monitoring were  strongly  supported  for  students  in  the  scaffolded  class.  Students  in  the  non-scaffolded  class  relied 
on  their  teacher  for  support  in  these  tasks,  rather  than  executing  the  processes  themselves.  This  is  illustrated  by  the 
role  the  classroom  teacher  assumed  for  his  students.  The  teacher’s  proposed  role,  that  of  “editor-in-chief,”  was 
demonstrated  while  his  students  were  writing  their  rough  and  final  drafts  of  their  articles.  Each  student  approached 
the  teacher  to  get  his  final  approval  on  each  sentence  or  paragraph  as  they  wrote  it.  This  contrasted  with  the 
scaffolded  class  where  the  students  received  their  guidance  and  support  from  the  scaffold  that  the  researcher 
provided  to  them.  Each  of  the  six  information  problem-solving  steps  was  provided  to  the  students  in  a booklet  that 
they  could  use  to  monitor  their  own  progress.  Additionally,  it  provided  them  with  an  explicit  process  to  follow  in 
completing  their  work.  Students  in  the  scaffolded  class  knew  exactly  what  to  do  while  students  in  the  non- 
scaffolded  class  had  to  ask  their  teacher  for  direction  at  each  step  of  the  writing  process. 

Researchers  have  claimed  that  students  who  can  successfully  analyze  tasks,  identify  strategies  for  task 
completion  (Palincsar  & Brown,  1981),  apply  problem-solving  strategies  in  appropriate  situations  (Eisenberg  & 
Berkowitz,  1990;  Palincsar  & Brown,  1981)  and  engage  in  self-monitoring  behaviors  (Bondy,  1984)  can  be 
considered  metacognitively  successful.  However  these  skills  are  difficult  for  children  to  acquire  (Brown,  1985). 
Bondy  (1984)  makes  several  recommendations  to  educators  interested  in  strengthening  students’  metacognitive 
skills.  These  include:  (1)  modeling  metacognitive  strategies  in  order  to  provide  students  with  an  understanding  of 
how  to  mentally  negotiate  difficult  cognitive  tasks,  (2)  requiring  students  to  keep  a daily  learning  log  to  shift  the 
cognitive  focus  from  product  to  process,  (3)  providing  instruction  in  self-questioning  techniques  to  redirect 
attention,  and  (4)  adapting  a learning  and  studying  model  to  assist  students  in  applying  strategies  in  a wide  variety  of 
situations.  The  relationship  between  the  IPS  model  and  metacognitive  skills  discussed  in  the  literature  was 
illustrated  through  the  data  acquired  in  the  current  study. 

Gradually,  students  in  the  scaffolded  class  began  to  understand  that  the  steps  of  the  IPS  model  could  help 
them  understand  where  they  were  supposed  to  be  at  the  end  of  each  day,  and  where  they  were  supposed  to  start  at 
the  beginning  of  the  next  day.  Journal  entries  confirmed  this.  Except  for  one  student  who  exhibited  extremely  high 
levels  of  anxiety  in  relation  to  the  assignments,  most  students  indicated  that  they  were  either  moderately  or  highly 
confident  that  they  knew  what  to  do  the  next  day.  Eisenberg  & Berkowitz  (1988,  1990)  argue  that  the  Big  Six 
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Information  Skills  is  a model  that  students  and  teachers  can  use  in  learning,  studying,  and  problem-solving 
situations.  As  such,  it  can  be  utilized  as  a learning  or  studying  model,  as  recommended  by  Bondy’s  (1984). 

The  second  possible  explanation  for  the  significant  differences  in  achievement  scores  was  the  time-on-task 
students  demonstrated  in  each  class.  Due  to  the  more  highly  defined  path  and  the  more  frequent  opportunities  to 
check  their  work,  students  in  the  scaffolded  class  spent  more  time  on  task,  rather  than  waiting  for  feedback  from 
their  teacher.  This  may  have  been  due  to  the  fact  that  the  procedures  in  the  scaffolded  class  provided  a pre-defined 
structure  for  students  to  follow  while  completing  their  study  activities.  This  structure  included  opportunities  for  peer 
feedback  and  personal  reflection  on  the  quality  of  their  work.  The  procedures  students  were  to  follow  were  provided 
in  written  form.  This  allowed  students  to  know  the  next  step  and  allowed  the  researcher  opportunities  to  shift 
research  responsibilities  back  to  the  students  by  having  a tangible  reference  point  available  during  discussions. 
Neither  the  students  nor  the  teacher  in  the  non-scaffolded  class  utilized  this  structure.  Thus,  these  students  needed 
much  more  hands-on  guidance  from  the  teacher. 

Students  from  both  classes  sought  out  either  the  researcher  or  teacher  for  guidance  during  the  activities. 
However,  students  in  the  non-Scaffolded  class  were  more  often  waiting  in  line  for  their  teacher’s  input,  while 
students  in  the  Scaffolded  would  frequently  form  groups  to  ask  the  researcher  a common  question.  The  classroom 
teacher  assumed  a high  degree  of  control  in  his  classroom.  This  contrasts  with  the  role  the  researcher  assumed  for 
her  class.  She  encouraged  the  students  in  her  class  to  assume  responsibility  for  the  completion  of  their  tasks, 
according  to  the  methodology  of  the  IPS.  Consequently,  engagement  with  the  topic  was  higher  for  the  scaffolded 
class.  The  increased  level  of  engagement  allowed  students  in  the  scaffolded  class  more  time  to  create  their  initial 
article  and  to  correct  any  mistakes  or  omissions  in  relation  to  the  content  of  their  articles.  This  supports  prior 
research  showing  a correlation  between  time  spent  engaged  in  instructional  tasks  and  increases  in  student 
achievement  (Doyle,  1983;  Montazemi  & Wang,  1995;  Van  Dusen  & Worthen,  1995). 

The  results  in  this  study  suggest  that  following  the  procedures  of  the  IPS  may  have  caused  students  to  shift 
their  mental  focus  from  a procedural  activity  to  an  internal  mental  process  (Bondy,  1984).  Previous  research  on 
children’s  thinking  processes  has  found  that  for  some  students  the  thinking  process  occurred  without  conscious 
direction  on  their  part  (McGregor,  1993).  The  students  in  the  scaffolded  class  demonstrated  that  they  were  more 
aware  of  how  their  thinking  affected  the  decisions  they  made.  One  student  told  the  researcher,  “I’m  finished  with 
the  Information  Seeking  Strategies  activity  and  will  start  my  Use  of  Information  activity  tomorrow.”  Many  students 
in  the  scaffolded  class  demonstrated  this  rudimentary  identification  of  thought  processes. 

A second  finding  that  is  significant  to  educators  is  that  the  IPS  might  have  an  impact  on  student 
engagement  with  the  topic.  The  benefits  of  increased  engagement  include  increased  time  on  task,  opportunities  for 
learning  through  repetitive  exposure  to  material,  readiness  for  higher-order  thinking  and  opportunities  for  richer 
evaluation  of  student  work  (Newmann,  1992).  By  focusing  the  students’  decision  making  on  whether  they  are 
satisfied  with  the  current  situation  and  what  they  can  do  to  change  that  situation,  IPS  strategies  increase  the 
opportunity  for  learners  to  become  engaged. 

Lastly,  the  IPS  process  did  not  increase  anxiety  levels  for  students  in  the  scaffolded  class.  Students  in  both 
classes  agreed  with  statements  indicating  generally  positive  feelings  toward  the  study  unit  as  a whole.  Additionally, 
the  classroom  teacher  liked  the  accountability  the  IPS  scaffold  provided  for  the  students  and  the  structure  it  provided 
to  students.  He  stated  that  he  would  be  willing  to  use  it  if  he  were  to  implement  another  unit  of  this  type  in  his 
classroom. 

When  interpreting  the  results  from  this  study,  readers  should  consider  that  due  to  the  unique  population  of 
participants  and  the  following  limitations  the  results  presented  are  not  generalizable  beyond  the  current  study.  The 
teaching  styles  of  the  two  adults  in  the  classroom  may  have  affected  the  results.  The  researcher  acted  as  a 
participant  in  the  study.  As  such,  her  biases  and  influences  must  be  considered  in  light  of  the  research  question. 
The  researcher  did  not  have  prior  knowledge  about  the  school  performance  of  the  students  in  the  Scaffolded  class. 
Therefore,  her  impressions  of  the  students  were  based  on  what  she  observed  during  the  implementation  of  the  study. 
The  classroom  teacher  had  spent  the  entire  school  year  with  the  students  and  had  formed  opinions  about  the  students 
based  on  their  performance  over  the  previous  six  months.  These  differences  in  opinions  concerning  the  students 
may  have  impacted  the  interactions  between  the  teacher  and  researcher  and  the  students  in  each  class. 

Implications 

Prior  research  has  found  that  one  of  the  essential  skills  students  must  possess  in  order  to  be  successful  in 
problem-based  learning  activities  is  metacognition  (Hill  & Hannafin,  1997;  Land  & Hannafin,  1997).  IPS  models 
act  as  metacognitive  scaffolds  that  support  students  while  they  become  more  adept  at  monitoring  their  own  thought 
processes  during  the  problem  solving  process.  The  structured  vocabulary  the  IPS  model  provides  allows  teachers 
and  students  to  label  behaviors  and  clarify  terminology,  two  activities  that  are  recommended  to  enhance 
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metacognitive  ability  in  students  (Costa,  1984).  Consequently,  an  unobservable  process  can  be  monitored  and 
tracked  through  a set  of  prescribed  steps  and  described  using  a standardized  vocabulary. 

In  addition  to  shifting  the  focus  of  student  metacognition  from  covert  to  overt,  implementing  the  IPS  model 
allows  students  to  spend  more  time  on  task  in  a problem-solving  situation.  Increasing  the  time  on  task  is  likely  to 
increase  student  achievement  (Doyle,  1983;  Montazemi  & Wang,  1995).  The  IPS  process  provides  a cognitive  map 
for  students  to  follow  as  they  solve  information-based  problems.  This  map  encourages  students  to  assume 
ownership  and  responsibility  for  their  problem-solving  process.  An  added  benefit  of  increased  time  on  task  is  that 
students  are  exposed  to  the  content  involved  with  the  problem  situation  more  frequently.  This  may  assist  students  in 
better  comprehending  the  information  relevant  to  their  problem. 

IPS  models  may  provide  overarching  processes  that  students  can  employ  in  a variety  of  learning  situations. 
The  benefit  of  strengthened  metacognitive  skills  is  that  students  can  then  apply  these  skills  to  a variety  of  learning 
situations  that  may  differ  from  the  area  in  which  the  process  was  initially  introduced  (Bondy,  1984). 

The  results  in  this  study  provide  support  for  a growing  body  of  research  suggesting  that  with  appropriate 
support,  students  can  succeed  at  complex,  learner  centered,  research-oriented  tasks  (Brush  & Saye,  2000;  Eisenberg 
& Berkowitz,  1998;  Hill  & Hannafin,  1997;  Land  & Hannafin,  1997;  Marchionini,  1989;  Perzylo  & Oliver,  1992). 
However,  implementing  a new  process  for  completing  activities  in  school  might  be  difficult  at  first.  Students  may 
be  uncomfortable  with  the  accountability  the  IPS  models  place  on  them.  By  introducing  the  process  in  small  steps 
and  infusing  IPS  vocabulary  throughout  the  school  day,  teachers  can  help  students  become  more  comfortable  and 
skilled  at  implementing  the  metacognitive  skills  that  are  supported  by  this  model. 

Conclusion 

Research  suggests  that  implementing  a process  approach  to  research  skills  can  be  effective  when  certain 
conditions  are  met  (Kuhlthau,  1993).  These  conditions  include  strong  team-based  planning  and  implementation 
activities,  an  emphasis  on  student  engagement,  and  the  presence  of  a collaborative  learning  environment.  The 
current  study  suggests  that  a specific  process,  the  Big  Six  Information  Skills  (Eisenberg  & Berkowitz,  1988),  might 
be  effectively  used  as  a metacognitive  scaffold  for  students  solving  information-based  problems. 
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Abstract 

The  purpose  of  this  paper  is  to  investigate  how  collaborative  mathematical  problem  solving  affects 
individual  mathematical  problem-solving  ability.  This  paper  begins  with  a review  of  the  applicable  theoretical 
notions  on  problem  solving,  and  on  collaboration  as  it  relates  to  problem  solving.  Next  it  explains  the  experiment ’s 
method  and  procedures,  and  then  goes  on  to  analyze  and  discuss  its  results.  Finally  it  presents  some  conclusions, 
along  with  a brief  discussion  of  the  project’s  implications  for  further  research.  The  results  show  no  quantitative 
differences  between  collaborative  and  individual  problem  solving.  However,  some  qualitative  differences  were 
evident. 

Introduction 

“Problem  solving”  has  been  regarded  and  designated  by  many  mathematics  educators  and  researchers  (see, 
for  example,  Goldin,  1982;  Lester,  1982;  Mayer,  1982;  Schoenfeld,  1982,  1985)  as  an  important  activity  in 
mathematics  learning.  Indicative  of  the  perceived  importance  of  this  topic  is  its  inclusion  by  the  National  Council  of 
Teachers  of  Mathematics  (NCTM)  as  a heading  in  its  new  standards  for  the  mathematics  curriculum,  which  list 
mathematics  as  problem  solving  along  with  mathematics  as  reasoning,  mathematics  as  connection,  an  mathematics 
as  communication  (NCTM,  1989). 

“Collaboration”  too  has  been  much  advocated  recently  as  a powerful  factor  both  in  learning  in  general  and 
in  problem  solving  in  particular.  This  advocacy  is  based  on  social  interaction  frameworks  (see,  for  example,  L.  S. 
Vygotsky,  1978),  and  is  supported  by  the  findings  of  research  experiments  (for  example,  Forman,  1981).  These 
results  confirm  that  for  many  types  of  problem  solving  the  results  of  collaboration  are  superior  to  those  of  individual 
efforts.  But  how  and  why  does  collaboration  function  so  effectively?  More  specifically,  what  are  the  mechanisms 
through  which  collaboration  promotes  problem-solving  abilities?  This  has  been  the  motivating  question  behind  my 
research  effort;  and  the  purpose  of  this  paper  is  to  provide  at  least  some  partial,  tentative  answers. 

In  order  to  gain  some  answers,  an  experiment  was  designed  and  conducted  that  would  enable  the 
experimenter  to  observe  both  individual  and  collaborative  problem-solving  activities  together,  and  to  compare  their 
results  in  terms  of  both  quantitative  and  qualitative  ethnographic  perspectives.  This  paper  will  begin  with  a review 
of  the  applicable  theoretical  notions  on  problem  solving,  and  on  collaboration  as  it  relates  to  problem  solving.  Next 
it  will  explain  the  experiment’s  method  and  procedures,  and  then  go  on  to  analyze  and  discuss  its  results.  Finally  it 
will  present  some  conclusions,  along  with  a brief  discussion  of  the  project’s  implications  for  further  research. 

Theoretical  Background 

It  is  not  too  overwhelming  to  say  that  the  human  civilization  is  a history  of  problem  solving  activities 
(Yang,  1994).  Often  in  our  daily  lives  we  find  ourselves  working  to  solve  problems  in  our  lives,  whether  these 
problems  are  big  or  small,  important  or  unimportant.  We  try  to  gain  knowledge  and  skills  both  from  school  and  in 
our  daily  lives  in  order  to  solve  these  problems.  For  example,  we  learn  mathematics  in  order  to  solve  those  problems 
related  to  number,  quantity,  shapes,  area,  volume,  and  so  on.  And  indeed,  “mathematical  problem  solving”  is  both 
the  main  goal  of  mathematics  learning  (Brodinsky,  1977;  LeBlance,  1977)  and  the  most  important  activity  of 
mathematics  learning  (Lester,  1980).  In  recent  years  advocacy  from  mathematics  educators  and  researchers  has 
resulted  in  increasing  attention  given  to  issues  concerning  mathematics  problem  solving  and  in  the  adoption  of  this 
activity  as  a major  focus  of  a new  set  of  standards  for  K-12  mathematics  curriculum. 

In  traditional  mathematical  problem-solving  activities  in  school  settings,  the  most  often  used  learning 
method  is  individual  problem  solving,  rather  than  collaboration.  Moreover,  collaboration,  often  regarded  as 
cheating,  is  commonly  prohibited  as  a mathematical  problem  solving  activity.  Mason  (1972)  has  the  following 
impressive  observation,  “We  isolate  students  by  pitting  them  against  each  other  competitively,  and  imposing  on 
them  a fierce  decorum  of  silence  and  regimentation”  (p.  6). 
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In  view  of  the  fact  that  collaboration  has  become  a popular,  effective,  and  efficient  form  of  learning  in  daily 
life  and  in  the  work  place,  and  has  been  advocated  as  a mode  of  working  style  to  meet  the  rapid  changes  and  big 
challenges  of  modem  society  (Senge,  1994),  it  seems  reasonable  to  suppose  that  the  mathematics  curriculum  should 
also  increase  its  acceptance  of  collaborative-oriented  problem  solving  in  order  to  prepare  the  learners  to  collaborate 
with  others  in  order  to  solve  problems  in  daily  life. 

The  significance  of  collaboration  can  be  understood  from  two  perspectives:  first  regarding  it  as  a practical 
phenomenon,  and  then  from  the  viewpoint  of  developmental  psychology.  In  the  following  paragraph  I shall  consider 
each  of  these  perspectives  in  more  detail. 

As  a practical  phenomenon,  collaboration,  or  “working  together”,  is  in  the  very  nature  of  society  (Mason, 
1972),  and  “inherent  in  everyday  interaction”  (Choi  & Hannaffin,  1995,  p.  62).  As  Forman  and  McPhail  (1993)  put 
it,  “The  everyday  lives  of  adults  are  full  of  complex  and  ill-defined  problems  that  require  high-level  reasoning  and 
organizational  skills.  These  problems  are  often  solved  in  collaboration  with  other  people.  For  example,  a husband, 
wife,  and  babysitter  may  need  to  coordinate  their  weekly  occupational  and  domestic  work  schedules  in  order  to 
supervise  one  or  more  young  children”  (p.  213). 

To  acknowledge  the  fact  that  all  human  progress  is  a result  of  collaboration  amounts  not  only  to  a simple 
affirmation  that  collaboration  is  the  most  common  mode  of  social  functioning  (Resnick,  1987),  but  also  to  an 
indication  of  its  status  as  the  most  magical  of  all  human  activities.  In  common-sense  terms,  progress  is  possible 
because  “collectively,  we  can  be  more  insightful,  more  intelligent  than  we  can  possibly  be  individually.  The  IQ  of 
the  team  can,  potentially,  be  much  greater  than  the  IQ  of  the  individuals”  (Senge,  1994,  p.  239). 

But  how  and  why  does  collaboration  function  so  universally,  so  effectually?  The  anthropologist  Edwin 
Hutchin  gives  us  an  example  that  lets  us  begin  to  see  how  collaboration  structures  the  efforts  of  a society,  the 
formation  of  knowledge,  and  even  the  nature  of  cognition  (in  Resnick,  1987).  In  Hutchin’s  view,  guiding  a ship  by 
triangulation  on  coastal  landmarks  entails  both  a complex  interaction  between  helmsman  and  lookouts  and  a 
collaborative  articulation  of  the  fabric  of  knowledge  itself,  involving  distance  experts  such  as  cartographers  and 
gyrocompass-builders.  Here  we  cannot  speak  of  collaboration  without  saying  something  about  the  knowledge- 
structures  involved,  nor  can  we  speak  of  the  required  knowledge  without  saying  something  about  the  collaborative 
processes  through  which  it  is  put  into  effect. 

When  we  look  at  the  social  deployment  of  knowledge,  we  find  it  inseparable  from  collaborative  processes. 
That  is  to  say,  social  leaming-at  any  rate,  social  learning  outside  of  school-takes  place  through  collaboration.  As 
Resnick  (1987)  remarks,  “much  activity  outside  school  is  socially  shared.  Work,  personal  life,  and  recreation  take 
place  within  social  systems,  and  each  person’s  ability  to  function  successfully  depends  on  what  others  do  and  how 
several  individuals’  mental  and  physical  performance  mesh”  (p.  41).  Even  if  we  choose  to  view  learning  as  an 
individual  process,  we  will  find  that  it  is  collaboration  that  provides  the  “context  in  which  supports  for,  constraints 
on,  and  challenges  to  an  individual’s  thinking  occur”  (Forman  & McPhail,  1993,  p.  213).  Yet  there  is  reason  to 
believe  that  knowledge  is  best  measured  as  the  collaborative  achievement  of  a group.  Barnes  and  Todd  (1977), 
studying  groups  engaged  solely  in  assigned  talk,  found  that  these  groups  attained  cognitive  levels  higher  than  those 
attained  by  individual  members.  Under  such  a notion,  knowledge  is  distributed  in  nature.  To  return  to  Resnick’s 
analogy  (1987)  from  Hutchin:  “No  individual  in  the  system  can  pilot  the  ship  alone.  The  knowledge  necessary  for 
successful  piloting  is  distributed  throughout  the  whole  system”  (p.  41).  Resnick’s  conclusion  is  that  most  work  and 
learning  in  a society  is  a matter  of  “shared  cognition,”  not  “individual  cognition.” 

L.  S.  Vygotsky  (1896-1934),  from  the  perspective  of  developmental  psychology,  provided  a conceptual 
framework  to  deal  with  this  question  through  his  analysis  of  the  relationship  between  social  interaction  and  higher 
mental  processes.  He  felt  that  higher  functions  originate  first  in  the  social  interactions,  before  they  are  integrated  into 
the  cognitive  structure  of  the  mind:  “Every  function  in  the  child’s  cultural  development  appears  twice:  first,  on  the 
social  level,  and  later,  on  the  individual  level;  first,  between  people  (interpsychological),  and  then  inside  the  child 
(intrapsychological).  This  applies  equally  to  voluntary  attention,  to  logical  memory,  and  to  the  formation  of 
concepts.  All  the  higher  functions  originate  as  actual  relations  between  human  individuals”  (Vygotsky,  1978,  p.  57). 

Moreover,  according  to  Vygotsky’s  view  of  development  as  a dialectical  process,  the  individual  cognitive 
structure  of  mind,  having  been  internalized  through  interaction  with  the  social  environment,  is  then  promoted  to  a 
state  in  which  it  will  itself  influence  the  social  environment.  So  there  is  an  ongoing  reciprocal  interaction,  a bi- 
directional relationship,  between  the  individual  and  the  social  environment.  On  the  one  side,  the  social  environment 
or  cultural  frame  within  which  the  individual  participates  and  interacts  plays  a crucial  role  in  the  development  of 
human  cognition;  on  the  other  side,  the  individual  self  is  also  an  essential  source  in  the  social  environment  for 
fostering  the  ongoing  reciprocal  processes  of  interaction. 

Vygotsky  further  proposed  that  any  learning  exploits  a “zone  of  proximal  development”  or  “ZPD.” 
Vygotsky  defined  ZPD  as  “the  distance  between  the  actual  developmental  level  as  determined  by  independent 
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problem  solving  and  the  level  of  potential  development  as  determined  through  problem  solving  under  adult  guidance 
or  in  collaboration  with  more  capable  peers”  (Vygotsky,  1978,  p.  86).  Vygotsky  observed  that  children  learning 
mathematics  seemed  to  be  using  the  ZPD  they  created  for  themselves  in  the  course  of  collaborative  problem-solving, 
in  such  a way  as  to  attain  higher  levels  of  achievement  than  they  would  have  been  able  to  manage  while  working 
along  (Yackel,  Cobb,  Wood,  Wheatley,  & Merkel,  1991).  ZPD  can  be  fostered  by  collaboration  with  adult  or  more 
competent  peers,  who  supply  a context  “in  which  support  for,  constraints  on,  and  challenges  to  an  individual’s 
thinking  occur”  (Forman  & McPhail,  1993,  p.  213).  Within  the  resultant  ZPD  a group  can  attain  cognitive  levels  not 
attained  by  individual  members,  leading  the  child  on  ahead  in  his  or  her  development. 

Purpose  of  the  Research 

The  above  section  has  explored  the  practical  phenomenon  of  collaboration  and  some  theoretical  notions 
from  developmental  psychology  about  why  collaboration  works.  These  perspectives  constitute  the  general 
background  underlying  the  author’s  interest  in  investigating  how  collaboration  might  affect  individual  ability  in 
mathematical  problem  solving.  More  specifically,  what  are  the  mechanisms  through  which  collaboration  promotes 
individual  mathematical  problem-solving  abilities?  To  carry  out  this  investigation  a research  experiment  was 
designed  in  terms  of  the  following  questions: 

• Is  individual  mathematical  problem-solving  ability  after  collaboration  significantly  different  from  individual 
mathematical  problem-solving  before  collaboration?  If  so,  what,  quantitatively  speaking,  are  the  differences? 
And  what,  qualitatively  speaking,  are  the  important  characteristics  of  these  differences? 

• Is  collaborative  mathematical  problem-solving  ability  significantly  different  from  individual  mathematical 
problem-solving  before  collaboration?  If  so,  what,  quantitatively  speaking,  are  the  differences?  And  what, 
qualitatively  speaking,  are  the  important  characteristics  of  these  differences? 

• Is  collaborative  mathematical  problem-solving  ability  significantly  different  from  individual  mathematical 
problem-solving  after  collaboration?  If  so,  what,  quantitatively  speaking,  are  the  differences?  And  what, 
qualitatively  speaking,  are  the  important  characteristics  of  these  differences? 

Method 

A quantitative  evaluation  and  a qualitative  ethnographical  investigation  were  conducted,  focusing  on  four 
students  in  the  same  4th  grade  mathematics  class  as  they  participated  in  two  steps  of  individual  problem-solving 
activities  and  one  step  of  collaborative  problem-solving  activity. 

Participants 

The  data-collection  phase  of  the  research  was  carried  out  using  students  from  a 4th-grade  class  in  an 
elementary  school  in  Taipei,  Taiwan,  Republic  of  China.  All  these  students  were  Chinese-Mandarin  speakers.  A 
group  of  four  students  out  of  a class  of  thirty-five  was  selected  to  participate  in  this  study.  This  group  of  four 
students  represented  four  different  levels  of  achievement:  high,  median-high,  median,  and  low.  The  selection 
method  for  this  class  was  as  follows: 

A copy  of  the  research  proposal  was  sent  to  the  school’s  research  office,  which  in  turn  solicited  teachers 
interested  in  letting  their  students  to  participate  in  the  project.  The  class  group  selected  for  this  study  was  one  taught 
by  one  of  the  interested  teachers  who  taught,  initiated,  and  guided  mathematics  collaborative  problem  solving  and 
discourse.  Because  the  participants  were  members  of  a class  group  that  was  already  familiar  with  collaborative 
problem  solving  and  discourse,  and  also  with  non-synchronous  thinking-aloud  skills  (Yang,  1994),  an  adaptation  of 
thinking-aloud  (Ericsson  & Simon,  1985),  the  research  project  did  not  necessitate  introducing  the  participants  to  any 
unfamiliar  models  of  collaborative  problem  solving.  No  problems  involving  the  establishment  of  a collaborative 
relationship  were  anticipated,  because  the  participating  students  had  practiced  collaborative  discourse  and  problem- 
solving since  they  were  in  the  third  grade. 

The  design  of  the  selection  of  participants  proposed  by  the  researcher  was:  All  of  the  students  in  the  class 
were  grouped  into  four  ranks,  based  on  their  fourth-grade  mathematics  achievements  level.  One  student  from  each 
rank  was  randomly  selected  by  means  of  a random  number  table.  Four  students,  representing  a mixture  of 
mathematics  achievement  levels,  high,  median-high,  median,  and  low,  were  assigned  to  each  group.  The  design  of 
the  selection  of  group  size  is  consistent  with  the  suggestions  of  the  way  of  grouping  in  cooperative  learning  (which 
suggests  small  group  size  of  3-5  persons)  (Johnson  & Johnson,  1994),  The  design  of  forming  group  members  is 
consistent  with  Vygotsky’s  ZPD  notion  which  suggests  heterogeneous  grouping  (Vygotksy,  1978). 

In  the  research  report,  the  participants’  responses  necessarily  be  described.  However,  no  real  names  were 
be  used.  Rather,  the  participant  with  high  level  achievement  was  given  by  a letter  “H,”  the  participant  with  median 
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high  level  achievement  was  given  by  a letter  “MH,”  the  participant  with  median  level  achievement  was  given  by  a 
letter  “M,”  and  the  participant  with  low  level  achievement  was  given  by  a letter  “L”. 

Materials  and  Instrumentation 

The  purpose  of  the  study  is  to  investigate  how  collaboration  might  affect  individual  ability  in  mathematical 
problem  solving.  A vital  preliminary  to  this  project  is  to  understand  the  context,  Jen's  Choice,  where  the  participants 
of  the  research  study  were  working  with  problem  solving.  Therefore,  there  is  a need  to  introduce  at  the  beginning  of 
this  section  about  Jen's  Choice. 

Jen  's  Choice 

Jen's  Choice  is  a Chinese-language  computer-anchored  4th-grade  mathematics  instruction  program, 
designed  and  developed  by  Dr.  Hsin-Yih  Shyu  (1997)  and  associates  at  Tamkang  University  in  Taiwan.  Based  on 
the  notions  of  anchored  instruction  and  situated  cognition  (CGTV,  1990),  it  provides  an  inquiry,  and  real  life 
learning  environment  and  authentic  tasks  for  learner  to  solve  mathematics  and  other  cross-subject  problems.  The 
story  describes  that  Jen,  a fifth  grader  was  accused  by  her  classmates  of  stealing  a watch  from  the  class  model 
student,  because  she  showed  her  watch  after  the  model  student  lost  her  watch.  In  order  to  find  the  giver  of  her  watch, 
Jen’s  grandmother,  to  approve  that  the  watch  was  really  given  by  her  grandmother,  Jen,  accompanied  by  her  three 
friends,  planned  and  took  a trip  to  visit  her  grandmother.  In  order  to  do  so,  they  had  to  encounter  several  plans  and 
decisions,  such  as  time  planning,  expense  planning,  among  others,  and  these  need  them  to  apply  4th-grade 
mathematics  and  across-subject  knowledge  to  solve  problems  and  plans. 

At  the  end  of  the  story,  two  types  of  problems,  one  type  is  about  life  problem,  the  other  is  about 
mathematics  problems,  are  provided  to  challenge  the  learner  who  view  this  program.  In  this  research  study,  only 
mathematics  problems  are  given  to  the  participants  to  solve.  The  mathematics  problems  contain  the  following  three 
mathematics  problems: 

Question  #1:  Up  to  now  (“now”  meaning  before  they  buy  tickets  for  the  rides),  how  much  money  does  each  person 
have  left? 

Question  #2:  At  the  Children’s  Recreation  Center,  if  everyone  wants  to  ride  the  same  rides,  then  how  should  they 
spend  the  remaining  money  on  tickets?  (Note  that  the  cheapest  ride  at  the  recreation  center  costs  three  tickets.) 
Question  #3:  Supposing  that  Jen’s  apartment  is  15  minutes  away  from  the  Tamsui  Station  bus  stop,  and  taking  into 
consideration  that  she  has  to  be  home  before  5:30  p.m.,  answer  the  following  questions. 

a)  What  is  the  longest  possible  time  they  can  stay  at  the  Children’s  Recreation  Center? 

b)  What  is  the  latest  possible  time  they  can  leave  the  Children’s  Recreation  Center?  (supposing  that  it  will  take 
them  10  minutes  to  wait  for  a bus  in  front  of  the  recreation  center) 

c)  Which  show  can  they  see  at  Tomorrow’s  World  Theater?  1 :00  p.m.,  2:00  p.m.,  or  4:00  p.m. 

Because  the  nature  of  these  three  mathematics  problems:  authentic  tasks,  complex  situations,  real-life, 
therefore,  it  makes  meaningful  that  Jen’s  Choice  was  used  as  instruments  for  test  problem  solving. 


The  research  experiment  consisted  of  three  steps.  In  step  1,  participants,  working  individually  on  PC 
computer  in  a separated  decent  room,  solved  mathematics  problems.  First,  each  of  the  participants  was  shown  Jen's 
Choice.  After  that,  three  mathematics  problems  from  this  program  were  given  to  each  individual  participant  to  solve 
using  paper  and  pencil.  The  participants  were  asked  to  show  their  mathematical  problem-solving  procedures,  and 
also  to  engage  in  non-synchronous  thinking  aloud  (Yang,  1994),  an  adaptation  of  thinking-aloud  method  (Ericsson 
& Simon,  1985)  Thinking  aloud  is  a research  method  that  involves  asking  each  participant  in  an  experiment  to  speak 
out  loudly,  explaining  as  fully  as  possible,  for  every  process  he/she  performs,  how,  why,  and  what  he/she  is  thinking 
and  doing.  This  is  a vital  method  for  investigating  a participant’s  thinking  processes.  However,  since  the  participant 
has  to  think  aloud  at  the  same  time  that  he/she  is  doing  a task,  this  procedure  could  interfere  with  the  participant’s 
thinking  and  acting.  Therefore,  the  method  of  non-synchronous  thinking  aloud,  was  employed:  participants  were 
requested  to  ‘think  aloud’,  not  while  performing,  but  a little  after  completion  of  each  segment  of  the  task,  in  order 
not  to  interrupt  the  participant’s  thinking  and  doing  the  task.  There  was  a facilitator  with  each  of  the  participants  to 
encourage  non-synchronous  thinking  aloud  and  to  remind  the  participants  not  to  forget  the  non-synchronous 
thinking  aloud. 

In  step  2 participants  undertook  group  collaborative  problem  solving.  One  hour  after  the  completion  of  step 
1,  the  four  participants  were  gathered  together  in  a decent  room  having  a PC  computer,  to  solve,  collaboratively,  the 
same  three  problems  they  had  solved  individually  in  step  1.  And  as  above,  they  were  asked  to  engage  in  non- 
synchronous  thinking  aloud  while,  working  as  a group,  they  worked  out  their  collaborative  approaches  to  the 
problems  with  paper  and  pencil. 
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In  step  3 participants  again  undertook  individual  problem  solving,  on  a PC  computer  in  a separated  decent 
room.  One  hour  after  the  completion  of  step  2,  each  of  the  four  students  was  asked  to  solve,  individually,  the 
identical  three  problems  again.  All  the  procedures  in  step  3 were  identical  to  those  in  step  1 . 

All  three  steps  of  the  experiment  were  video  recorded  and  audio  taped.  Cordless  microphones  were 
distributed  to  each  participant  to  ensure  that  each  participant’s  voice  was  clearly  recorded  and  clearly  heard  by  the 
others.  These  three  groups  of  written  solutions  and  video-recorded  non-synchronous  thinking-aloud  protocols  were 
compared  and  analyzed  for  quantitative  and  qualitative  differences.  Quantitative  comparisons  were  made  by 
evaluating  the  degree  of  correctness  of  the  different  final  answers  given  to  each  of  the  three  questions  during  each  of 
the  three  steps.  For  qualitative  comparison,  these  protocols  were  analyzed  in  terms  of  the  following  five  components 
(Yang,  1994): 

1.  Problem  perception,  understanding  and  representation:  evaluates  the  problem  solver’s  understanding  of 
what  the  problem  is,  and  what  the  situation  of  the  problem  is;  evaluates  the  problem  solver’s  ability  to 
draw  a figure  to  introduce  situational  notation. 

2.  Mathematics  knowledge  and  concepts:  evaluates  the  participant’s  ability  to  access  and  apply  the 
mathematical  knowledge  and  concepts  that  are  relevant  to  the  problem. 

3.  Devising  and  carry  out  the  plan:  evaluates  the  problem  solver’s  ability  to  devise  a plan  based  on  the 
problem,  and  to  implement  the  plan  step  by  step. 

4.  Consciousness  and  control  of  the  plan:  evaluates  the  problem  solver’s  ability  to  consciously  control  what 
he/she  is  doing — to  judge  if  it  is  meaningful  or  reasonable,  and  to  adjust  it  when  appropriate. 

5.  Looking  back:  evaluates  the  problem  solver’s  ability  to  check  the  process  of  what  he/she  has  done, 
identifying  and  correcting  any  mistakes  or  inappropriate  solutions. 

Results  and  Discussion 

Figure  1 shows  a summary  of  the  total  number  of  right  answers  for  each  of  the  three  steps  of  problem 


Participant 

Step 

H | MH  M L 

Total  no.  of  right  answers  (Total  no.  of  questions  = 3) 

Step  1 (individual  p-s) 

0 0 0 | 0 

Step  2 (collaborative  p-s) 

% 

Step  3 (individual  p-s) 

~o  fo  o To 

Fig . 7.  Number  of  right  answers  achieved  by  the  participants  in  each  of  the  three  steps  of  the  experiment 


It  seems  that  the  results  for  collaborative  problem  solving  were  not  significantly  better  than  those  for  individual 
problem  solving.  Furthermore,  collaborative  problem-solving  experience  did  not  seem  to  facilitate  later  individual 
problem  solving.  But  these  conclusions,  representing  strictly  quantitative  results,  are  inherently  limited  in  scope; 
they  reveal  nothing  about  the  actual  processes  through  which  the  participants  constructed  their  understanding.  At 
this  point  the  learning  process  remains  a ‘black  box.’  An  additional,  qualitative,  analysis  is  required  in  order  to 
identify  and  evaluate  the  changes  in  thinking  that  took  place  over  the  three  steps  of  problem-solving  activity. 
Because  the  space  limitations  in  this  paper,  the  findings  of  this  additional  analysis  are  only  briefly  summarized  here 
(Figures  2 to  5): 
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Fig.  2.  Components  of  problem  solving  that  participant  H performs 
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Fig.  3.  Components  of  problem  solving  that  participant  MH performs 
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Fig.  4.  Components  of  problem  solving  that  participant  M performs 
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Fig.  5.  Components  of  problem  solving  that  participant  L performs 


One  of  the  reasons  that  all  of  the  participants  had  very  low  achievement  in  solving  the  three  problems  was 
that  they  did  not  fully  understand  the  problems  when  individually  solving  these  problems.  For  example,  when 
Question  #1,  “Up  to  now  (meaning  before  they  bought  tickets  for  the  rides),  how  much  money  does  each  person 
have  left?”  can  serve  as  an  example  here.  In  the  first  step,  the  participants  misunderstood  it  as  meaning  either  ‘how 
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much  money  did  each  of  them  have  before  they  started  the  journey?’  or  ‘how  much  total  money  did  they, 
collectively,  have  left  before  paying  for  the  rides?’  Accordingly,  they  devised  wrong  plans,  carried  out  wrong  plans, 
and  arrived  at  wrong  things. 

Doubtless  the  relatively  high  degree  of  complexity  of  the  problems  contributed  to  these  frequent 
misunderstandings.  Traditional  educational  approaches  had  accustomed  the  participants  to  less  complex  problems, 
for  which  all  they  needed  was  simply  to  put  all  of  the  numbers  together  without  thinking  too  deeply.  Thus,  when 
they  met  with  this  experiment’s  much  more  complex  problems,  they  were  totally  lost  and  confused,  and  tended  to 
revert  to  just  putting  the  numbers  together.  In  such  complex  and  challenging  situations,  collaboration  does  not  seem 
to  help,  as  can  be  seen  in  the  results  from  the  second  and  third  problems.  Even  though  the  participants  came  to  the 
experiment  with  considerable  prior  experience  in  collaborative  problem  solving  and  related  mathematics  knowledge 
and  concepts  needed  for  solving  problems,  collaboration  at  this  time  did  not  facilitate  successful  solutions. 

Also,  deficiencies  in  the  categories  of  consciousness  and  looking  back  resulted  most  participants  reaching 
wrong  answers  for  almost  all  of  the  problems,  even  though  they  performed  reasonable  problem-solving  procedures. 

However,  collaboration  did  facilitate  control  and  monitoring  of  the  collaborative  problem-solving  process. 
From  the  protocols,  we  were  able  to  see  that  solutions  and  plans  were  revised  due  to  the  reminders  from  peer 
participants:  when  some  peers  were  unable  to  find  the  errors,  others,  having  found  them,  reminded  those  peers  to  be 
cautious  and  to  rethink;  the  resultant  back-and-forth  arguments  often  resulted  in  reasonable  solutions.  This  striking 
collaborative  phenomenon  may  represent  the  most  essential  aspect  of  the  collaborative  function.  As  Forman  (1981) 
said,  “Both  Vygotsky  and  Piaget  would  agree  that  social  interaction  provides  the  individual  with  feedback  about  his 
own  thoughts  and  action  which  enables  him  to  reflect  upon  and  modify  his  behavior”  (p.  2).  Thus,  The  potential 
effectiveness  of  collaborative  control  in  the  problem-solving  process  is  evident,  for  example,  in  the  results  from 
problem  #1.  Here,  none  of  the  participants  were  able,  individually,  before  or  after  collaboration,  to  solve  the 
problem  correctly,  or  even  to  find  the  correct  data  to  on  which  to  do  the  calculation;  but  in  collaboration,  they  were 
able  to  identify  almost  all  the  data  needed  to  do  the  calculation. 

Although  the  above  findings  do  not  support  the  conclusion  that  collaboration  significantly  promotes  later 
individual  problem-solving  ability,  the  qualitative  comparison  data  show  that  collaboration  does  have  an  effect  on 
collaborative  problem-solving  strategies. 

Conclusions 

The  aim  of  this  research  effort  was  to  develop  a deeper  understanding  of  how  collaboration  affect 
individual  mathematical  problem-solving  ability.  In  this  research,  we  investigated  how  individual  and  collaborative 
mathematical  problem  solving  took  place  in  instructional  contexts  when  anchored  in  authentic  problem  situations. 
The  results  of  the  research  help  us  to  see  the  different  ways  in  which  individual  and  collaborative  problem  solving 
approaches  contributed  to  the  participants’  problem-solving  strategies.  These  findings  remain  tentative,  due  to  the 
participants’  unfamiliarity  with  the  style  of  the  problem  items  (in  particular,  their  high  complexity),  the  small  sample 
size,  and  the  restricted  components  of  observation.  Further  research  designed  to  circumvent  these  limitations  should 
achieve  more  complete  and  definitive  results. 
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During  the  last  50  years,  the  major  tenets  of  most  educational  subjects  areas  have  undergone  little  change; 
however,  the  characteristics  of  the  student  population  and  methods  of  teaching  the  concepts  have  evolved.  The 
student  population  has  been  transformed  by  the  differing  ethnic,  disabled,  and  cultural  backgrounds  that  have 
entered  the  educational  system.  The  methods  of  teaching  have  been  impacted  by  the  multimedia  software  and 
hardware. 

The  movement  of  including  students  with  disabilities  in  the  general  education  curriculum,  specifically 
students  with  learning  disabilities  (LD),  behavioral  and/or  emotional  disorders  (EBD),  promoted  by  The  Individuals 
with  Disabilities  Education  Act  (IDEA)  has  created  a persistent  challenge  for  the  teachers  who  work  with  them. 
These  students  may  have  near  average  or  above  intellectual  abilities,  but  disengage  when  in  the  general  education 
setting  for  a variety  of  reasons  (Kauffman,  2001;  Koyangi  & Gaines,  1993).  The  tight  spot  special  educators  find 
themselves  in  is  how  to  connect  these  students  to  the  curriculum,  age-appropriate  peers,  and  teachers  while  still 
ensuring  effective  learning. 

Even  though  many  of  these  students  have  the  intellectual  power,  they  cannot  function  in  the  general 
education  curriculum  due  to  skill  deficits  they  show  in  performance.  These  perceived  intellectual  gaps  complicate 
the  connection  between  these  students  and  their  same-age  peers  because  of  the  lower  level  of  schoolwork  they 
appear  to  be  doing.  If  they  are  to  remain  in  the  inclusive  setting  these  skills  must  be  integrated  with  their  daily 
learning  of  new  skills.  They  attend  programs  where  the  emphasis  is  on  behavior  management  and  social  adjustment 
ahead  of  academics  and  vocational  preparation  (Knitzer,  Steinber,  & Fleisch,  1990).  They  are  underserved  (Lewis, 
Chard,  & Scott,  1994)  and  placed  in  more  restrictive  environments  due  to  the  availability  of  adequate  public  school 
education  programs  (Kauffman  & Lloyd,  1992).  Dodge  and  Coie  (1990)  found  that  the  strength  of  the  bond  between 
students  with  special  needs  and  their  same-age  peers  resulted  from  how  competent  students  with  special  needs  felt  at 
school.  This  competency  also  influenced  the  students  with  special  needs  level  of  self-concept,  self-esteem,  and  self- 
identity.  That  is,  it  is  necessary  to  develop  lessons  that  engage  this  population  and  allow  students  to  perform  close  to 
grade  level  for  their  affective  as  well  as  academic  success. 

The  demand  for  educational  accountability  by  policy  makers  and  constituents  increases  the  pressure  for 
general  education  teachers  who  already  have  onus  for  teaching  diverse  populations  of  students.  The  additional 
responsibility  of  addressing  state  content  standards  and  assessment  while  simultaneously  addressing  the  affective 
needs  of  these  students  overshadows  their  ability  to  embrace  a student  with  special  needs  into  their  classroom.  Thus, 
special  education  must  focus  support  in  the  modification  and  accommodation  of  the  student’s  academic  needs  in 
order  to  free  the  general  education  teacher  to  address  their  affective  needs  and  their  social  inclusion. 

Teacher  preparation  programs  have  recognized  their  obligation  to  provide  solutions  for  the  dilemmas  their 
teachers  face  once  in  the  inclusive  educational  environment.  The  changing  demands  required  by  both  general  and 
special  education  teachers  at  the  K-12  level  are  being  addressed.  One  of  these  solutions  is  to  provide  access  to 
technological  innovations  that  help  teachers  meet  the  needs  of  a diverse  population,  complete  the  job  quickly  and 
efficiently,  and  have  the  ability  to  integrate  best  practices  into  the  interface  design  and  lesson  content. 

The  study  described  here  was  designed  to  contribute  to  the  field’s  knowledge  of  the  use  of  technology  to 
support  modifications  for  students  with  LD  and  EBD  in  the  general  education  curriculum.  The  process  and  outcomes 
of  thirty  teachers  in  a Special  Education  Master  degree  program  using  a template  design  system  to  modify  lessons 
for  students  with  learning,  behavioral,  and/or  emotional  disabilities  in  a general  education  curricular  subject  of 
science  or  language  art  was  examined  for  efficiency,  effectiveness,  and  appeal. 

The  researcher  applied  Gagne,  Briggs,  and  Wager’s  (1992)  condition  of  learning  model  to  teaching  the 
template  use  to  the  special  education  teachers.  Gagne’s  five  identified  categories  of  learning  are  verbal  information, 
intellectual  skills,  cognitive  strategies,  motor  skills,  and  attitudes.  The  Kemp  instructional  system  design  (Kemp, 
1997)  identifies  nine  elements  to  step  teachers  through  developing  their  modified  lesson  using  the  template.  The 
Jerrold  Kemp  Design  Model  shown  in  Figure  1 takes  a holistic  approach  by  considering  factors  in  the  learning 
environment  and  includes  learner  characteristics,  subject  analysis,  learning  objectives,  teaching  activities,  resources 
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(computers,  books,  etc.),  support  services  and  evaluation.  Gagne’s  categories  integrate  nicely  with  the  Kemp  Design 
model.  The  entire  process  includes  the  best  practice  of  teach,  assess,  teach,  as  well  as  being  subject  to  continuous 
revision. 


Figure  L The  Kemp  ISD  Model 


Method 

Participants 

The  participants  in  this  study  were  30  teachers  pursuing  a Masters  degree  for  Colorado  licensure  in  Special 
Education.  Fifteen  teachers  taught  in  juvenile  facilities  and  15  taught  in  public  schools.  All  30  teachers  taught  at  the 
secondary  grade  levels.  Each  of  these  teachers  was  responsible  for  teaching  using  the  Colorado  State  Content 
Standards  and  preparing  students  to  take  the  Colorado  Student  Assessment  (CSAP).  The  students  these  teachers 
taught  qualified  for  special  education  services  in  the  primary  categories  of  learning  disabilities  and  emotional 
disturbances  (designated  in  Colorado  as  Moderate  Needs  and  Significantly  Identifiable  Emotional  Disturbance 
(SIED),  respectively).  Eighteen  of  the  master  level  teachers  were  earning  general  education  licensure  while 
simultaneously  working  under  a state  temporary  teaching  eligibility  (TTE)  for  special  education  licensure.  That  is, 
12  had  a previously  earned  B.A.  in  Education  while  18  did  not  have  any  previous  teaching  coursework  and  were  on 
Emergency  Teaching  Licensure.  In  addition,  all  of  the  participants  were  earning  the  master  degree  in  special 
education  while  teaching  with  a TTE.  See  Table  1 for  details  of  the  educational  qualifications  of  participants. 


Table  1.  Teacher  Education  Qualifications 


Juvenile  Facility  Teachers  (N=15) 

Public  School  Teachers  (N=15) 

Teach  Grades  8-10 

7 

0 

Teach  Grades  9-12 

8 

15 

Earned  B.A.  in  education 

3 

0 — Special  Education  — 1 

2 - Elementary  — 5 

1 — Physical  Education—  1 

Secondary  (Science, 

0 — English,  or  Math)  — 2 

9 

Temporary  Teaching  Eligibility 

15 

15 

Emergency  Education  Licensure 

12 

6 

There  were  6 males  and  9 females  employed  in  juvenile  facilities  and  2 males  and  13  females  employed  in 
the  public  schools  with  an  average  age  of  27  years  and  31  years,  respectively.  See  Table  2 for  a participants’ 
personal  characteristics  report  regarding  teachers’  gender,  age,  and  years  of  experience  in  teaching. 
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Table  2.  Teacher  Demographics 


Juvenile  Facility  Teachers 

Public  School  Teachers 

Gender 

Age 

# of  Years  Teaching 

Gender 

Age 

# of  Years  Teaching 

1 

M 

24 

1 

1 

M 

23 

1 

2 

M 

26 

1 

2 

M 

26 

1 

3 

M 

28 

2 

3 

F 

26 

3 

4 

M 

30 

2 

4 

F 

26 

1 

5 

M 

33 

4 

5 

F 

29 

4 

6 

M 

35 

6 

6 

F 

30 

6 

7 

F 

25 

2 

7 

F 

30 

5 

8 

F 

26 

1 

8 

F 

26 

1 

9 

F 

27 

1 

9 

F 

26 

1 

10 

F 

29 

1 

10 

F 

27 

3 

11 

F 

31 

1 

11 

F 

28 

3 

12 

F 

31 

2 

12 

F 

29 

2 

13 

F 

31 

2 

13 

F 

30 

6 

14 

F 

32 

3 

14 

F 

32 

12 

15 

F 

35 

10 

15 

F 

32 

10 

Measures  and  Procedures 

To  begin  the  study  each  of  the  teachers  were  given  two  pre-instruments.  The  first  assessed  each  teacher’s 
level  of  technological  sophistication  and  level  of  technophobia.  Technological  sophistication  was  measured  by  a 
researcher-developed  questionnaire.  The  questionnaire  solicited  information  on  teachers’  use  of  consumer 
technology  (e.g.,  video-cassette  recorder,  automated  banking,  computer/video  games,  home  care  software), 
academic  technology  (e.g.,  word  processing,  programming  use,  library/research  technology,  classroom  presentation 
packages/technology),  and  technology  ownership.  Technophobia  was  measured  by  level  of  computer  anxiety.  The 
Computer  Anxiety  Rating  Scale  (Form  C)  (CARS-C)  (Rosen,  Sears,  & Weil,  1988)  contains  20  items  on  a five-point 
scale  that  when  scored  derived  a level  of  “no  technophobia,”  “moderate  technophobia.”  or  “high  technophobia.” 

The  second  evaluation  tool  examined  the  teachers’  ability  to  develop  lesson  plans  that  included 
modifications/adaptations  for  students  with  specified  characteristics  of  learning  disabilities  and  emotional/behavioral 
disabilities.  According  to  the  Colorado  State  Department  (CDE,  2000)  special  education  teachers  are  to  link  state 
content  standards  to  each  student’s  IEP  annual  goals.  Therefore,  teachers  were  also  required  to  include  the  state 
content  standard  they  were  addressing  in  their  lesson  plan.  Teachers  were  in  need  of  support  that  allowed  flexibility. 
Thus,  the  dependent  variables  were  (1)  efficiency  of  the  Web-based  lesson  template,  (2)  effectiveness  of  the 
template  use  on  technical  and  instructional  component  inclusion  in  lesson  plans,  and  (3)  the  appeal  of  the  template 
use  to  special  and  general  education  teachers. 

The  training  (independent  variable)  began  with  each  participant  being  taught  how  to  develop  necessary 
modifications  and/or  adaptations  to  promote  success  of  students  with  affective  and  academic  disabilities  on  daily 
general  education  assignments  and  assessments.  The  areas  of  emphasis  for  modifications  were  lesson  activities, 
required  readings,  and  evaluations.  In  addition  to  teaching  possible  modifications,  each  teacher  was  taught  how  to 
write  lesson  goals  that  introduced  the  broad  topic,  learning  objectives  that  included  performance-based  terms,  action 
verbs,  links  to  the  goal,  and  were  measurable  and  observable. 

Each  teacher  was  then  instructed  in  the  use  of  a web-editing  program  (Microsoft  Front  Page)  and  Teaching- 
Not-Teching  (T-N-T)  (Murry,  1998),  a web-based  lesson  template.  Each  instruction  period  lasted  2 hours  with  Vi 
hour  for  questions  and  guided  hands-on  experimentation.  T-N-T  was  designed  to  include  each  of  the  7 components 
the  teachers  had  been  taught  to  use  and  include  in  a lesson  plan  for  their  students  with  special  needs.  The  T-N-T 
template  included  navigation  for  goals,  objectives,  readings,  activities,  evaluation,  glossary  or  chat  room,  and 
teacher  e-mail  link.  See  Figure  2 for  a screen  capture  of  the  navigation  found  on  each  page  of  the  T-N-T  lesson 
template. 
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Figure  2.  T-N-T  Template  Navigation 


After  selecting  their  topic  area  (Science 
or  Language  Arts)  the  teachers  were  provided 
time-log  worksheets  to  track  time  that  they  spent 
thinking,  tinkering,  developing,  and  collecting 
items  for  their  Web-based  lesson.  They  were 
then  instructed  to  begin  the  development  of  their 
first  Web-based  lesson  using  the  steps  listed 
below. 

Step  1 : Consider  the  learner  characteristics; 
Step  2:  Matching  characteristics  to  the 

instructional  problem  the  lesson  presented; 

Step  3:  Articulate  what  the  student  will 

accomplish  in  this  lesson  using  performance- 
based  terms; 

Step  4:  Write  measurable  goal(s)  and 

objectives  for  the  specific  lesson  using  evidence 
from  the  students’  Individualized  Education  Plan 
(IEP); 

Step  5:  Use  the  subject  matter  expert  (SME) 
(e.g.,  the  general  education  Science  or  Language 
Arts  teacher)  to  provide  ideas  on  materials, 
graphics,  and  other  resources; 

Step  6:  Identify  and  determine  which  modifications  and/or  accommodations  are  necessary  for  success; 

Step  7:  Identify  and  collect  the  online  readings  and  activities  that  relate  back  to  the  lesson  goal(s)  and 

objectives; 

Step  8:  Develop  your  lesson,  planning  the  delivery  and  content  outline; 

Step  9:  Develop  evaluation  instruction  to  conduct  the  assessment  for  your  lesson; 

Over  the  following  10-week  period,  the  teachers  developed  8 other  Web-based  lessons  using  Front  Page  and  T-N-T 
for  use  with  students  needing  parallel  curriculum  adaptation/modifications  in  the  general  education  classroom. 

A. 

Results 

The  same  battery  of  measurement  instruments  was  administered  to  all  30  teachers  before  and  after  the 
training  program.  The  performance  of  the  teachers  on  the  pretest  and  posttest  measures  is  shown  in  Table  3.  Mean 
group  raw  gain  scores  were  observed  on  each  of  the  lesson  plan  instructional  and  technical  components  (see 
columns  1 and  2).  To  determine  whether  the  gains  were  educationally  and  statistically  significant,  the  researchers 
subtracted  the  pretest  score  from  the  posttest  score  to  yield  a gain  score.  The  mean  gain  on  each  component  for  the 
total  sample  is  shown  in  column  3 of  Table  3. 

Table  3.  Group  Gain  Scores  on  Dependent  Variables 


Group  Pretest  Score 

Group  Posttest  Score 

Group  Gain  Score 

Efficiency 

Lesson  Development 

Time 

347  minutes 

56  minutes 

291  minutes 

Effectiveness 
Instructional  (7  total) 

2.1 

6.4 

4.3 

Technical  (3  total) 

.5 

2.9 

2.4 
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Appeal 

Future  Intended  Use 

15 

27 

12 

Technophobia 

25 

8 

17 

The  T-N-T  template  efficiency  was  defined  as  amount  of  time  gained  between  the  first  Web-based  lesson 
and  the  ninth  lesson  development.  Teachers  kept  track  of  the  minutes  they  spent  thinking  about  content  of  the 
lesson,  tinkering  with  Front  Page  and  T-N-T,  surfing  the  World  Wide  Web  for  graphics,  collecting  audio,  video,  and 
other  resources.  The  average  amount  of  time  spent  on  the  development  of  the  first  lesson  was  347  minutes  compared 
to  55  minutes  on  the  ninth  Web-based  lesson.  See  Table  4 for  a detailed  comparison  of  each  participant's 
development  time  between  the  first  and  ninth  lesson. 


Table  4.  Time  Log  Comparison  Between  lessons 


Effectiveness  for  the  purpose  of  this  study  was  defined  as  the  inclusion  and  functionality  of  technical  and 
instructional  components  in  the  Web-based  lesson  plans.  The  T-N-T  lesson  template  navigation  included  each  of  the 
instructional  components  with  the  exception  of  a Content  Standard  link,  thereby  making  it  difficult  for  a teacher  to 
ignore  the  need  to  include  them.  The  technical  components  were  also  included  in  the  template  design;  however,  the 
lesson  developer  could  alter  the  functionality.  See  Table  5 for  pretest  and  posttest  results  of  each  participant's 
inclusion  of  instructional  components  and  Table  6 for  pretest  and  posttest  results  of  functionality  of  the  lesson 
technical  components. 
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Table  5.  Pre  and  Post  Instructional  Scores  for  Each  Participant 


Table  5.  Pre  and  Post  Technical  Scores  for  Each  Participant 


The  dependent  variable  of  appeal  was  characterized  by  the  intended  future  use  of  the  template  by  the 
special  and  general  education  teachers  and  by  the  change  in  pretest  to  posttest  technophobia  scores.  The  results  of 
intended  future  use  survey  and  the  pretest  posttest  technophobia  scores  are  reported  in  Tables  7 and  8,  respectively. 
Table  7.  Pretest  and  Posttest  use  and  Intended  future  use 
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Pretest 

Posttest 

Yes  N (%) 

Yes  N (%) 

Use  Web-based  lesson  template  to  adapt  general  education 
lessons  for  students  with  special  needs 

0 (0%) 

27  (90  %) 

Intend  to  expand  the  variety  of  technology  used  in  my  future 
teaching 

2 (6  %) 

27  (90  %) 

Table  8.  Pretest  and  Posttest  Technophobia  Scores 


Pretest 

Posttest 

N (%) 

N (%) 

High  technophobia 

6 (2%) 

2 (.6  %) 

Moderate  technophobia 

13  (4  %) 

3(1%) 

No  technophobia 

1 1 (3.6%) 

25  (8  %) 

Discussion 

Technophobia  is  a major  concern  in  the  educational  arena.  Technophobia  has  been  categorized  as  the 
number  one  reason  teachers  fail  to  integrate  technology  in  to  the  classroom.  Utilization  of  the  T-N-T  template  was 
investigated  as  a means  to  reduce  technophobia  to  a manageable  level  and  thereby  allow  teachers  to  integrate 
technology  into  the  classroom  in  the  form  of  Web-based  instruction. 

As  a result  of  incorporating  the  use  of  T-N-T  participant  performance  was  substantially  improved  in  several 
areas  of  concern.  Technophobia  was  reduced  in  two  categories  (Moderate  and  High)  with  many  students  reporting 
moving  into  the  No  Technophobia  category.  Three  students,  who  scored  high  in  technophobia  during  the  pre-test, 
showed  little  or  no  response  whatever  in  the  learning  curve.  The  same  students  did  not  score  well  on  the  pre  and 
post  for  template  effectiveness,  efficiency  and  appeal.  This  may  have  been  due  to  attitude,  age,  or  lack  of  motivation 
to  use  computer-based  technologies.  These  students  were  the  oldest  in  either  of  the  groups. 

Average  lesson  development  time  decreased  from  347  minutes  to  55minutes  by  the  ninth  lesson.  The 
number  of  instructional  strategies/components  included  in  lessons  developed  by  the  participants  increased 
dramatically.  This  increase  is  encouraging  in  that  earlier  teacher  instructional  behavior  research  has  indicated  that 
the  discriminating  factor  between  novice  and  expert  teachers  was  that  novices  who  did  not  start  teaching  with 
effective  lesson  development  skills  did  not  acquire  them  simply  as  a result  of  experience  (Ayers,  1983;  Housner  & 
Griffey,  1994;  Medley,  1980). 

Functionality  of  the  technical  components  included  in  the  Web-based  lessons  developed  by  the  participants 
went  from  0 for  many  participants  to  the  highest  possible  post-test  score  of  three.  The  use  of  the  template  made  this 
increase  possible;  however,  the  participants  could  have  easily  made  the  components  nonfunctional.  The  fact  that  so 
many  of  the  teachers  succeeded  with  the  template  provides  a solution  to  the  findings  of  McCormack  and  Jones 
(1998).  They  found  few  educators  possessed  the  required  integrative  ability  to  combine  the  technical  knowledge  and 
educational  principles  to  construct  effective  Web-based  educational  environments.  The  template  allows  teachers  to 
circumvent  the  necessary  learning  for  technical  technology  skills  while  capitalizing  on  their  emergent  instructional 
skills. 

Limitations 

The  researchers  used  the  one-group  pretest-posttest  design  in  this  experiment  because  the  special  education 
teachers  were  expected  to  provide  modifications/adaptations  of  assignments  for  students  in  the  general  education 
classroom.  The  absence  of  a control  group  did  not  pose  a serious  threat  to  the  internal  validity  of  the  experiment, 
however,  because  the  researchers  were  able  to  safely  assume  that  expected  pretest-posttest  gains  due  to  extraneous 
factors  would  be  minimal  or  nonexistent. 

Conclusion 

The  use  of  a Web-based  lesson  template  increased  the  potential  for  special  education  teachers  to  effectively 
support  the  inclusion  of  students  with  EBD  and/or  LD  in  the  general  education  curriculum.  As  teachers  become 
increasingly  more  efficient  in  Web-based  lesson  design  they  will  transform  education.  The  use  of  the  T-N-T 
template  will  promote  lessons  that  are  derived  from  measurable  student  goals  and  objectives,  activities  that  are 
linked  to  the  goals  and  objectives,  and  quizzes  that  assess  the  stated  learning  to  take  place.  The  results  of  this  study 
promote  the  thinking  that  teachers  make  selections  based  upon  availability  and  immediate  usefulness  instead  of 
applying  the  concepts  or  principles  on  teacher  effectiveness  (Guskey,  Huberman,  & Michael,  1995;  Lortie,  1975) 
and  will  support  the  use  of  applications  that  enhance  immediate  usefulness  and  teacher  effectiveness.  It  will  also 
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encourage  the  use  of  technology  for  communication  between  teacher  and  students.  The  Office  of  Technology 
Assessment  declared  that  technology  was  generally  used  in  classrooms  for  low-level  cognitive  and  instructional 
tasks  (1995).  This  study  refutes  the  very  idea.  “We  have  the  potential  to  do  great  things  with  technology  in  our 
schools,  but  it  is  a potential  still  largely  unrealized”  (Riley,  1999,  p.  9). 
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Preface 


For  the  twenty-third  year,  the  Research  and  Theory  Division  of  the  Association  for 
Educational  Communications  and  Technology  (AECT)  is  sponsoring  the  publication  of 
these  Proceedings.  Papers  published  in  this  volume  were  presented  at  the  National  AECT 
Convention  in  Denver,  CO.  A limited  quantity  of  these  Proceedings  were  printed  and 
sold.  It  is  also  available  on  microfiche  through  the  Educational  Resources  Clearinghouse 
(ERIC)  system. 

For  the  first  time,  these  Proceedings  are  published  in  two  volumes.  This  volume  contains 
papers  primarily  dealing  with  instruction  and  training  issues.  Papers  dealing  primarily 
with  research  and  development  are  contained  in  the  companion  volume,  which  also 
contains  over  60  papers. 

REFEREEING  PROCESS:  Papers  selected  for  presentation  at  the  AECT  Convention  and 
included  in  these  Proceedings  were  subjected  to  a reviewing  process.  All  references  to 
authorship  were  removed  from  proposals  before  they  were  submitted  to  referees  for 
review.  Approximately  fifty  percent  of  the  manuscripts  submitted  for  consideration  were 
selected  for  presentation  at  the  convention  and  for  publication  in  these  Proceedings.  The 
papers  contained  in  this  document  represent  some  of  the  most  current  thinking  in 
educational  communications  and  technology. 


M.  R.  Simonson 
Editor 
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Collaborative  Design  and  Implementation  of 
a Large  University's  Web-based  Course 


Chong  Ho  Yu 
Angel-Jannasch-Pennell 
Samuel  A.  DiGangi 
Ruvi  Wijesuriya 
Arizona  State  University 


Abstract 

This  paper  discusses  issues  relating  to  the  design „ development , and  delivery  processes  of  multimedia 
modules  such  as  Macromedia  Flash®,  Shockwave  movies ®,  and  Quicktime  movies®.  These  modules 
were  employed  to  teach  an  undergraduate  plant  biology > class  at  a large  southwest  university.  Each 
medium  has  different  strengths  and  weaknesses . Their  proper  use  resulted  from  the  collaboration 
among  the  content  experts , instructional  designers,  and  multimedia  developers . 

Equipped  with  modem  web  technologies,  instructional  designers  have  abundant  resources  to 
deliver  courses  based  in  a multimedia  and  rich  with  interactivity.  However,  design  is  only  one  of  several 
crucial  factors  of  a successful  web-based  course.  No  matter  how  rich  the  media  features  are,  obstacles 
during  implementation  hinder  students  from  effective  learning.  For  example,  faculty  without  experience  in 
multimedia  development  and  distance  education  may  under-estimate  the  required  cost  and  resources,  which 
may  result  in  a delay  of  delivery  and  frustration.  In  addition,  inexperienced  Web  developers  might  not 
realize  the  diversity  of  user  computers.  Web  pages  look  good  in  a particular  platform,  a particular  browser, 
and  a particular  setting  might  look  different  in  other  computers.  Further,  digital  movie  is  said  to  be  a useful 
illustration  tool,  however,  lack  of  appropriate  plug-ins  and  bandwidth  may  make  the  media  unusable. 
VanHom  (2000)  realized  that  bandwidth  limitation  would  worsen  the  digital  divide.  Unfortunately,  Web- 
based  multimedia  developers  might  not  be  aware  of  this  limitation.  In  light  of  these  potential  obstacles,  this 
paper  discusses  how  a major  university  designed,  developed  and  delivered  a feature-rich  web-based  course 
for  plant  biology.  It  was  found  that  in  this  case  the  design  and  delivery  processes  were  more  problematic 
than  development.  Solutions  derived  from  our  experience  are  suggested. 

Course  Objectives 

Plant  biology  108  fulfills  the  Natural  Science  General  Studies  Course  Requirements  (SI  & S2).  In 
keeping  with  the  criteria  of  S I/S2  courses,  the  following  are  objectives  for  Concepts  in  Plant  Biology 

1 . To  provide  a substantial  introduction  to  the  fundamental  behavior  of  matter  and  energy  as  it  relates 
to  plants  and  to  the  plants’  role  in  the  biosphere. 

2.  To  introduce  the  students  to  the  scientific  method  and  to  have  them  gain  experience  with 
application  of  the  scientific  method  to  botanical  problems.  It  is  hoped  that  with  such  experiences  the 
student  will  be  able  to  use  the  concept  of  the  scientific  method  for  solving  problems  in  everyday  life. 

3 . To  gain  an  appreciation  and  understanding  of  how  plants  work  so  that  they  may  be  manipulated  to 
help  solve  such  world  problems  as  hunger,  pollution,  and  global  warming. 

In  order  to  fulfill  the  preceding  objectives,  use  of  multimedia  animation  were  considered  because 
several  concepts  in  biology  are  process-based;  it  is  more  instructionally  beneficial  to  illustrate  those 
concepts  in  animation  than  plain  text.  The  implementation  of  this  multimedia  web-based  course  was 
divided  into  three  stages:  Design,  development,  and  delivery  (see  Figure  1).  In  the  design  stage,  the  focus 
centered  around  curriculum  design,  content  development,  and  media  acquisition.  Instructors,  including 
faculty  and  graduate  assistants  of  plant  biology,  served  as  the  content  experts.  Based  on  their  input, 
instructional  designers  suggested  the  appropriate  media.  In  the  development  stage,  a positive  feedback  loop 
was  established  among  faculty,  instructional  designers,  and  multimedia  developers.  Since  the  Web  content 
could  not  updated  easily,  in  the  delivery  stage  a formative  evaluation  scheme  was  implemented.  Students 
were  encouraged  to  give  comments  and  to  report  bugs  to  faculty,  graduate  assistants,  and  technical  support 


personnel.  The  feedback  was  re-directed  to  the  instructional  designers  and  multimedia  developers  for 
corrections  and  enhancements  of  the  courseware.  Each  stage  will  be  discussed  in  detail  in  the  next  section. 


Figure  1. 


Allocation  of  both  human  and  material  resources  is  crucial  to  the  development  of  a Web-based 
class.  Besides  specifying  the  task  involved  in  each  stage,  the  team  also  estimated  the  workload  of  each 
group  in  each  stage  (see  Figure  2).  This  estimation  set  a reasonable  expectation  on  each  team  member  to 
ensure  a smooth  collaboration. 

Figure  2.  Resource  allocation 
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Design 


PLB108  is  a collaborative  project  between  the  Biology  department  and  Instruction  Support  group. 
Faculty  and  graduate  assistants  in  the  Plant  Biology  department  served  as  the  content  experts.  Staff  in 
Instruction  Support  served  as  instructional  designers  and  multimedia  developers  (design  team).  The  design 
team  advised  and  assisted  with  course  design,  interface  design,  and  storyboarding.  The  faculty  worked 
closely  with  the  Instructional  designers  to  take  ideas  for  concepts  and  convert  them  into  a manner  that  best 
utilizes  the  multimedia  delivery  system.  Based  on  the  input  from  the  content  experts,  the  design  team 
identified  three  media  for  serving  different  purposes,  namely,  MacroMedia  Flash  (MacroMedia,  2000a), 
Director  shockwave  (MacroMedia,  2000b),  and  QuickTime  (Apple,  Inc, .2000).  Their  capabilities, 
liabilities,  and  proper  applications  are  described  below: 

Flash 


Flash  can  use  vector-based  graphics  and  therefore  its  file  size  is  much  smaller.  Unlike  bitmapped 
graphics  that  are  composed  of  pixels,  vector-based  graphics  define  the  composition  of  an  image  by 
algorithms.  A typical  animated  module  made  in  Flash  is  as  small  as  10K. 

Another  advantage  of  vector  graphics  is  their  scalability.  Keeping  the  appearance  of  a Web  page 
consistent  is  a challenge  to  Webmaster  because  monitor  size,  resolution,  and  browser  size  vary  from 
computer  to  computer.  Vector-based  graphics  answer  this  challenge.  No  matter  what  the  monitor  size  and 
the  resolution  are,  vector-based  graphics  are  displayed  at  a pre-specified  percentage,  and  they  will  resize 
themselves  according  to  the  browser  size. 

As  mentioned  before,  consistency  and  bandwidth  limitation  are  considered  major  hindrances  from 
course  delivery.  Vector-based  graphics  are  definitely  one  of  the  ideal  media  for  the  Web. 

However,  vector-based  graphs  carry  less  information  than  their  bitmapped  counterparts. 
Therefore,  they  are  best-suited  for  drawing-based  images  rather  than  photo-realistic  images.  It  doesn’t  mean 
that  Flash  cannot  import  pixel-based  graphics  for  photo-realistic  illustration,  but  including  bitmapped 
images  will  definitely  inflate  the  file  size.  Moreover,  Flash  lacks  the  interactive  features  and  programming 
capabilities  as  that  of  Director. 

In  the  course,  Flash  is  primarily  used  for  modularized  presentations.  When  a complex  process  is 
presented,  a continuous  and  linear  animation  such  as  QuickTime  may  cause  confusion  among  learners.  An 
interactive  step-animation  is  designed  with  the  logical  break-down  of  the  process.  For  example,  the  Meiosis 
process  was  broken  down  into  three  steps.  In  each  step,  the  animation  is  co-presented  with  descriptive  text. 
The  learner  has  the  freedom  to  replay  a particular  step  of  animation  (Figure  3). 


Figure  3,  Meiosis  process  presented  in  Flash 


£«ch  tU»nd  Is  c«lM  a chromatid. 


DMA  Is  replicated  so  that  each 
chromosome  is  composed  pf  two 
strands. 


Step  1 


Chromatids  become  visible  at  the 
start  of  meiosis. 


This  approach  is  also  useful  to  illustrate  complex  structure.  In  the  Flash  module  displaying  a 
flower  structure,  different  components  are  showed  in  different  steps,  but  the  transition  between  steps 
overlay  translucent  components  so  that  students  understand  how  different  parts  of  a flower  are  related  to 
each  other.  In  addition,  Flash  can  also  be  used  to  associate  geographical  regions  with  biomes,  which  are 
worldwide  groups  of  similar  ecosystems  that  can  be  defined  by  their  major  vegetation  type.  In  a Flash 
module,  the  user  can  click  on  several  hot  spots  of  a map  to  zoom  in  the  region  and  photos  of  the  region  will 
be  revealed  (see  Figure  4).  Without  an  interactive  module,  the  student  may  find  it  difficult  to  go  back  and 
forth  between  a map  and  photos. 

Figure  4.  Map  of  biomes 


Shockwave  is  made  by  Macromedia  Director,  a multimedia  authoring  system  with  a powerful 
programming  language,  Lingo.  Therefore,  complicated  modules  such  as  highly  interactive  tutorials  are 
better  created  in  Director.  Multimedia  modules  may  involve  a time-based  process  (e.g.  growth  of  a plant, 
environmental  change),  a structural  relationship  (e.g.  cell  structure),  or  both.  Every  multimedia  authoring 
package  adopts  some  type  of  analogy,  which  is  temporal  or/and  spatial  oriented,  to  display  the 


Shockwave 
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programming  environment.  For  example,  HyperCard,  HyperStudio  (Knowledge  Adventure,  Inc.,  2000), 
and  SuperCard  (IncWell,  DMG,  Inc.,  2000),  obviously,  use  a card  analogy  with  an  emphasis  on  spatial 
structure,  in  which  different  layers  and  objects  represent  different  functions.  Authorware  (Macromedia, 
2000c)  uses  a flow-chart  analogy  with  a focus  on  temporal  transition,  in  which  icons  "flow"  along  the 
decision  tree.  Director  uses  a movie  frame  and  channel  analogy.  This  programming  environment 
incorporates  both  temporal  (frame)  and  spatial  (channel)  dimensions.  Therefore,  it  is  considered  more 
powerful  than  other  authoring  packages,  which  use  either  temporal  only  or  spatial  only  metaphor.  However, 
since  graphics  in  Director  are  pixel-based  and  thus  the  final  product,  Shockwave,  may  be  bandwidth- 
consuming. 

In  this  course,  shockwave  is  used  for  interactive  tutorials  that  require  user  interactions.  For 
example,  in  the  illustration  of  natural  selection,  users  are  asked  to  drag  a dark  moth  and  a light  moth  to  a 
light-colored  tree.  Later  on,  the  same  moths  are  dragged  to  a dark  tree  darkened  by  pollution.  Before  the 
pollution,  the  light  moth  is  more  likely  to  survive  for  its  protective  color  in  relation  to  the  tree.  After  the 
pollution,  the  survival  chances  tip  toward  the  dark  moth  (Figure  5).  The  objective  of  this  exercise  is  to  let 
learners  see  how  environment  affects  natural  selection.  Although  this  concept  can  be  illustrated  by  text,  the 
camouflaging  function  of  moth’s  color  is  more  dramatic  to  learners  when  they  actively  move  the  moths 
from  one  background  to  another.  This  drag-and-drop  approach  is  also  used  for  testing  purposes.  For 
example,  after  students  learned  the  lesson  on  herbaceous  stem  anatomy,  they  were  asked  to  identify  the 
internal  organization  of  a stem  by  dragging  the  text  into  the  right  position.  The  exercise  has  a built-in 
correcting  mechanism.  If  the  student  fails  to  drop  the  text  into  the  right  place,  the  text  will  re\ert  to  the 
original  position  and  thus  the  student  has  to  start  it  over  until  all  components  are  correctly  identified  (see 
Figure  6). 

Figure  5.  The  Peppered  moth  presented  in  Shockwave 


5 


517 


Figure  6,  Stem  anatomy  exercise  presented  in  Shockwave 
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QuickTime 

The  strength  of  QuickTime  is  its  ability  to  show  realistic  movies  within  a low  bandwidth  because 
certain  third-party  software  utilities  such  as  Media  Cleaner  Pro  (Terran,  Inc.,  2000)  are  able  to  compress 
QuickTime  files  without  losing  viewable  quality.  One  of  the  drawbacks  is  that  QuickTime  does  not  have 
many  interactive  features.  Among  the  three  chosen  media,  QuickTime  is  the  most  bandwidth-intensive. 

QuickTime  is  a proper  medium  for  realistic  movies.  For  example,  many  biology  students  may  not 
have  a chance  to  use  a high-powered  electronic  microscope  to  observe  objects  at  the  molecular  level  such 
as  how  a new  life  is  formed  through  the  fertilization  of  an  egg  by  a sperm,  and  how  the  movement  of  a cell 
(see  Figure  7).  This  web-based  course  includes  QuickTime  movies,  which  were  converted  from  footage 
taken  from  microscopes.  In  addition,  QuickTime  can  be  used  for  illustrating  a time-lapsed  process  such  as 
the  growth  of  a plant.  The  purpose  of  this  QuickTime  illustration  is  to  explain  that  each  organism  has  a 
finite  size  that  it  can  achieve  (see  Figure  8). 

Figure  7.  Movement  of  a cell  showed  in  QuickTime 
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Figure  8.  A growth  process  illustrated  in  QuickTime 


Another  use  of  digital  video  is  the  re-creation  of  historical  events.  Reading  text  about  history  may 
be  dry,  however,  dramatization  of  history  by  actors  and  actresses  gain  students’  attention.  In  this  class, 
Mendel  and  Darwin,  prominent  figures  in  biology,  come  alive  in  digital  video  interview. 

Software  and  hardware  requirements  were  imposed  on  registered  students.  All  the  preceding 
multimedia  modules  were  designed  to  run  on  the  combination  of  these  specific  software  and  hardware 
configurations.  The  minimum  requirements  are: 

• Windows  95/98/NT  or  Mac  OS  7.5 

• Multimedia  Pentium  or  PowerPC 

• 28.8  modem 

• 800X600  resolution,  256  colors  (8-bit) 

• Netscape  4.0  or  Internet  Explorer  4.0  or  AOL  4.0 

• QuickTime  Player  3.0 

• Shockwave/Flash  Player  7.2 

The  design  team  was  aware  that  students  might  not  have  QuickTime  or/and  Shockwave/Flash 
plug-ins.  Technical  assistance  to  students  will  be  discussed  in  the  section  of  delivery. 

Development 

At  the  development  stage,  instructional  designers  worked  with  multimedia  developers  to  convert 
the  storyboards  into  multimedia  and  upload  the  course  to  the  web.  With  each  release,  the  faculty  proofread 
the  media  and  provided  feedback  and  changes  if  necessary.  To  simulate  a realistic  learning  environment 
for  beta-testing,  testers  accessed  the  web  content  through  a dial-up  modem  and  viewed  them  in  a 15-inch 
monitor.  It  was  found  that  download  time  of  QuickTime  movies  was  excessive.  To  counteract  this  problem, 
QuickTime  movies  were  burned  into  a compact  disc  and  offered  to  students  as  an  alternative.  To  increase 
the  user-friendliness  of  the  CD,  a front  end  written  in  MacroMedia  Director  was  inserted  so  that  users  could 
easily  navigate  across  movies.  Taking  bandwidth  into  consideration,  the  development  team  had  decided  to 
convert  QuickTime  movies  to  QuickTime  streaming  and  Real  streaming  movies  in  the  next  release.  The 
difference  between  a digital  movie  and  a digital  streaming  movie  is  that  the  latter  can  play  almost 
immediately  while  more  signals  are  being  “streamed”  to  the  destination. 

The  copyright  issue  was  a major  concern  during  the  development  process.  Besides  using  royalty- 
free  images,  the  development  team  created  many  drawings  and  diagrams.  Approximately  half  of  the 
development  time  was  spent  in  creating  original  artwork  for  the  course. 
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Delivery 


The  course  was  delivered  through  Blackboard’s  Courselnfo,  which  has  built-in  features  for  Web- 
based  courses  such  as  login,  quizzes,  grade  book,  chat  room,  bulletin  board,  user  access  tracking,  and  many 
others.  Students  were  required  to  log  in,  and  their  movement  within  the  course  website  could  be  tracked. 
Tracking  website  traffic  enables  the  network  administrators  and  WebMasters  to  identify  the  "rush  hours" 
and  thus  to  choose  a better  time  for  updating  webpages.  For  example,  the  user  log  (see  Figure  9)  clearly 
indicates  that  during  lunar  hours  and  in  the  evening  (5-6  p.m.)  the  server  received  most  hits  from  users. 

Figure  9.  Usage  of  Website  by  hour  of  the  day 
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Despite  that  hardware  and  software  requirements  were  specified  prior  to  the  course,  several 
students  were  not  able  to  fully  access  the  course  materials  due  to  the  absence  of  a proper  Web  browser  and 
plug-ins.  This  was  anticipated  and  a technical  support  team  had  stood  by  to  provide  assistance  in  upgrading 
the  browser  and  installing  plug-ins.  The  technical  support  service  was  accessible  by  both  email  and 
telephone. 

Summary 

Initially,  the  design  process  and  development  process  were  labor-intensive.  It  is  important  to 
structure  the  materials  during  the  design  process  so  that  later  modifications  are  minimal  for  fiiture  courses. 
Moreover,  when  the  course  is  well-structured  with  rich  content  and  media,  based  on  the  input  from 
experienced  faculty,  a graduate  teaching  assistant  is  able  to  deliver  the  course  seamlessly.  As  a result, 
experienced  faculty  can  be  released  from  teaching  introductory  classes  and  hence  concentrate  on  research 
and  teaching  of  upper  division  courses. 

Examples  of  multimedia  resources  mentioned  in  this  article  are  available  at 
http://is.asu.edu/plblQ8.  Readers  are  encouraged  to  explore  the  posted  modules  and  give  us  feedback. 
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Learning  System  Design  Considerations  in  Creating  an  Online 

Learning  Environment 


Introduction 

This  paper  describes  the  design  of  a web-based  learning  environment  for  leadership  facilitators  in 
a U.S.  military  organization.  The  overall  aim  of  this  project  was  to  design  a prototype  of  an  online  learning 
environment  that  supports  leadership  facilitators’  knowledge  development  in  the  content  area  of 
motivation.  This  learning  environment  was  designed  to  provide  new  learning  opportunities  for  facilitators 
to  build  content  knowledge  through  interactive  learning  activities  and  experiences  with  other  members  of 
their  own  community.  In  order  for  this  learning  and  performance  environment  to  continually  improve  and 
be  effective  in  the  long  run,  issues  concerning  diffusion  of  the  technology  and  ongoing  learning  community 
development  should  be  addressed. 

The  iterative  design  phases  used  to  develop  this  web-based  application  are  illustrated  in  Figure  1 
below.  The  processes  of  analysis,  design,  and  evaluation  are  often  thought  of  as  unique,  stand-alone 
events.  This  project  was  managed  as  a concurrent  solution  design  process  incorporating  elements  of 
strategic  thinking,  change  creation,  work  environment  design,  and  rapid  prototyping.  The  front-end  analysis 
and  impact  evaluation  were  built  into  the  design  process. 


The  use  of  this  iterative  solution  design  process  allowed  for  increased  efficiency  and  effectiveness 
in  making  ongoing  refinements  and  improvements  to  the  prototype.  The  designers  utilized  the  learning 
environment  design  principles  identified  in  the  next  section  to  drive  the  major  tasks  and  to  assure  project 
quality. 

Additionally,  actions  considered  necessary  for  ongoing  support  of  the  web-based  environment 
were  summarized  in  two  major  areas:  1)  Diffusion  of  the  technology  into  the  facilitator  community,  and  2) 
Continuous  improvement  and  adaptation  of  the  site  by  the  community. 

Key  learning  environment  design  principles  and  concepts 

The  project  team  agreed  upon  three  broad  areas  of  learning  environment  design.  These 
components  address  a wide  array  of  solution  analysis,  design,  development,  implementation,  evaluation 
and  diffusion  issues.  For  the  sake  of  brevity,  these  principles  have  been  listed,  but  not  fully  elaborated 
upon,  below.  These  principles  are  useful  as  guides  to  match  project  needs  with  learning  environment 
features. 


Scott  Schaffer 
Florida  State  University 
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Re-Design 


Figure  1.  Iterative  Design  Phases 
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Analyzing 

• Identify  people  who  are  accountable  for  the  results  of  learners  and  engage  them  in  the  design  goal 
setting  process. 

• Identify  components  and  levels  of  the  organizational  system  where  performance-based  results  are 
dependent  on  learning  achievement. 

• Determine  measures  of  success  of  learning  and  performance  on  an  individual  and  organizational 
basis. 

• Plan  how  you  will  evaluate  this  success  (pre-post  learning). 

• Identify  non-training,  work  environment  factors  that  could  contribute  to  the  success  or  failure  of 
the  web-based  learning  environment. 

• Create  a plan  for  diffusion  and  adaptation  of  the  learning  environment  within  the  organization. 

• Create  an  awareness  and  knowledge  of  the  analysis  process. 


Designing 

• Create  learning  objectives  from  performance  objectives,  based  on  task,  job  and/or  performance 
analyses. 

• Create  activities  that  gain  the  learners’  attention  by  engaging  them  in  the  learning  process. 

• Create  activities  that  show  learners  how  the  instruction  will  be  of  benefit  to  them. 

• Provide  ways  for  the  learners  to  practice  learning  activities  and  get  feedback  on  their  performance. 

• Create  ways  for  learners  to  assess  their  own  performance  in  a meaningful  way  that  simulates  the 
actual  performance  as  close  as  possible. 

• Create  ways  for  learners  to  access  a common  knowledge  base  of  resources  on  a given  topic. 

• Provide  ways  for  learners  to  interact  with  instructors  and  with  other  learners. 

• Provide  ways  for  learners  to  gain  multiple  perspectives  on  a particular  problem  solving  approach. 

• Provide  a means  for  learners  to  access  knowledge  based  on  previous  learners’  experiences. 

• Engage  members  of  the  learning  community  or  organization  in  adaptation  of  the  learning 
environment  for  better  fit  within  the  current  culture. 


Continuously  Improving 

• Collect  data  to  measure  the  success  of  learners  and  specific  learning  environment  design 
processes. 

• Identify  non-training,  work  environment  factors  that  could  contribute  to  the  success  or  failure  of 
the  web-based  learning  environment. 

• Adapt  learning  design  processes  / adapt  environment  to  optimize  performance  of  learners. 

• Collect  data  to  measure  the  success  of  performers  and  the  organization  in  meeting  goals  as  related 
to  learning  activities  and  objectives. 


These  key  design  principles  were  used  as  the  foundation  for  a pre-prototype  evaluation  plan.  The 
plan  included  questions  derived  from  a variety  of  performance  improvement  theories  and  models.  A 
solution  design  decision  aid  assisted  in  evaluating  the  effectiveness  of  the  learning  and  performance 
environment.  Tables  1 through  5 are  excerpts  of  this  decision  aid. 
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r Table  1:  Strategic  Organizational  Context  (adapted  from  Ely;  Kaufrnan;  Rogers)  Evaluation 
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Strategic  Linkages 

■ How  are  project  objectives  linked  to  the  job,  work  center,  and  organizational  missions  and  vision? 

■ Has  a continuous  improvement  plan  including  impact  evaluation  been  completed  for  this  project? 

Innovation-Change  Adoption 

■ Is  there  a primary  stakeholder,  and  owner  of  this  project  that  will  support  its  adoption  and  diffusion? 

■ What  are  barriers  that  may  prevent  long-term  or  continued  success  of  the  project? 


J..  Table  2:  Organization- Work  Center  Context  (adaptedjfrom^ 

Evaluation  Probes  > •: . 

Expectations  and  Feedback 

■ Do  performers  know  what  is  expected  of  them  on  the  job?  Do  they  know  when  they  have  done  the  job 

correctly? 

Tools  and  Information 

11  Will  supporting  documentation,  job  aids,  and  other  performance  support  be  available? 

■ Is  there  time  in  the  work  schedule  for  performers  to  use  new  skills  learned  in  training?  (TT) 

Rewards  and  Incentives 

" Is  completion  of  tasks  rewarded  or  punished? 


P Table  3:  Performer  Context  (adapted  froni  Kelletf Wedman  <fe  Graham)  Evaluation  Probes 

Motivation  and  Selfc  Concept 

■ Do  workers  want  to  do  good  work? 

• Do  they  monitor  their  own  performance? 

Performance  Capacity 

■ Are  workers  physically  able  to  complete  job  tasks? 

■ What  degree  of  flexibility  with  respect  to  work  pace,  structure,  and  organization  is 

required  of  workers? 

Skills  and  Knowledge 

■ Do  workers  have  adequate  knowledge  and  skills  to  do  the  job  well? 

■ Which  skills  taught  in  training  are  not  being  used  on  the  job?  Why  aren’t  they  being 

used?  (TT) 


Table  ^Instructional  Design  (adapted  from  Dick  and  Carey;  Keller)  Evaluation  Probes 

Practice/Assessment: 

■ Does  practice  seem  relevant  to  on-the-job  expectations  and  performances? 

B Do  practice  items/assessments  match  objectives? 

Feedback: 

■ Does  feedback  promote  learner  satisfaction  with  the  learning  experience? 

■ Does  feedback  encourage  further  exploration?  How  could  exploration  be  further 
encouraged? 

Interactivity: 

■ If  appropriate  to  objectives,  is  interaction  among  learners  supported? 


Table  5:  User-Computer  Interface  (adapted  from  Nielsen)  Evaluation  Probes 

Visibility  of  system  status 

9 Does  the  learner  know  where  he/she  can/should  go  next? 

Match  between  system  and  the  real  world 

9 Does  the  online  learning  environment  represent  or  support  the  work  environment? 

Consistency  and  standards 

* Is  there  a standard  convention  used  to  identify  key  areas  of  the  site? 

Aesthetic  and  minimalist 

■ Is  information  presented  progressively  from  more  general  to  more  specific,  thus  allowing 
learners  to  drill  down  to  in-depth  information  as  necessary? 


To  accomplish  the  objectives  of  this  particular  project,  the  iterative  design  phases  (performance 
analysi  s,  prototype  design,  and  prototype  re-design)  were  integrated  with  the  design  principles  to  produce  a 
functional  prototype.  A review  of  this  integration  process  and  recommendations  for  implementation  of  the 
prototype  follow. 


Performance  Analysis 

Performance  analysis  was  a process  of  defining  the  results  expected  by  the  leadership  community 
and  the  facilitator  practices  that  could  lead  to  achievement  of  those  results.  Current  and  desired  facilitator 
results,  practices,  and  attitudes  were  analyzed  during  this  phase.  Measurable  results,  critical  performance 
and  learning  measures,  achievable  facilitator  practices,  and  work  environment  factors  and  conditions  that 
support  or  hinder  performance  were  profiled.  Prototype  design  began  after  performance  objectives  and  the 
flow  of  information  in  the  leadership  facilitator  community  were  identified.  The  results  of  analysis  were 
used  to  guide  the  planning  for  the  evaluation,  design  and  implementation  of  the  learning  and  performance 
environment  solution. 


Prototype  Design  and  Re-design  (Continuous  Improvement) 

Participatory  design  processes  are  effective  in  building  stakeholder  and  user  commitment  to  the 
success  of  application  systems  through  their  active  participation  in  the  analysis  of  requirements  and  the 
specification  ofthe  system  design.  For  this  project,  each  design  iteration  and  improvement  was  based  on 
discussions  with  key  sponsors,  stakeholders  and  potential  users.  Participants  and  designers  collaborated  to 
develop  a potentially  valuable  product,  identify  possible  product  improvements,  and  generally  provide  one 
another  with  guidance  in  the  ongoing  design  of  the  product. 


During  this  concurrent  design  and  evaluation  process,  interviews  and  focus  groups  were  conducted 
during  which  visual  prototypes  clarified  concepts  that  were  being  discussed.  Initial  prototypes  were 
PowerPoint  slides,  which  were  transformed  into  a prototype  web  site  using  simple  WYSIWYG  web  design 
tools.  To  assist  in  this  transformation,  members  of  the  target  audience  provided  initial  feedback  regarding 
the  site  look  and  feel,  and  overall  concept.  Following  initial  feedback,  a “joint  application  design”  meeting 
was  held  with  sponsors  (designers,  project  managers,  directors)  during  which  the  first  relatively  primitive 
web-based  site  was  reviewed  live.  The  purpose  of  this  meeting  was  to  collect  data  to  assist  in  design 
decision-making.  This  review  followed  a somewhat  orderly  process  with  a set  of  structured  interview 
questions  accompanying  the  review  of  each  screen.  A major  goal  of  this  design  iteration  was  to  ensure  that 
the  organizational  learning  and  performance  objectives  of  the  site  were  addressed. 


Following  this  meeting,  revisions  were  made  accordingly  in  order  to  prepare  the  site  for  usability 
testing  / evaluation.  The  evaluation  process  included,  but  was  not  limited  to,  assessment  ofthe  aesthetic 
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quality  of  the  interface,  and  following  redesign,  assessment  of  the  content,  learning  activities,  and 
achievement  of  objectives. 

The  first  level  of  evaluation  was  with  facilitators  at  a large  training  unit  with  more  than  100 
facilitators.  Upon  familiarizing  the  facilitators  with  the  basic  goals  of  the  site,  they  were  provided  the  URL 
and  asked  to  provide  feedback  (we  have  been  requested  to  not  release  the  results  of  these  evaluation 
processes  by  the  client). 


Following  this  process,  the  evaluation  processes  listed  below  were  also  completed: 

• Learner/Performer  Evaluation  * A survey  was  placed  on  the  website  to  gather  input  from 
facilitators  regarding  their  reactions  to  site  content  and  learning  activities,  and  suggestions  for 
improvement.  Users  also  had  the  opportunity  to  provide  feedback  on  the  site  discussion 
board. 

• Individual  interviews  with  novice  users  were  conducted  to  observe  a typical  user  interaction 
and  resolve  any  barriers  to  navigation  and  learning.  The  user  went  through  each  activity  and 
made  suggestions  for  the  content  and  layout  of  the  site. 

Specific  strategies  and  recommendations  related  to  utilization  and  ongoing  support  of  the 
prototype  that  was  designed  and  delivered  to  the  client  were  proposed.  These  strategies  and 
recommendations  were  focused  in  two  key  areas:  1 ) Adopting  and  diffusing  the  product  within  the  target 
organization;  and  2)  Continuously  improving  the  design  and  usefulness  of  the  current  prototype  once  it  has 
been  adopted  by  the  target  organization. 

Adopting  and  Diffusing  the  Product:  Implementation  Strategies 

The  most  well  designed  products  often  go  unused  or  unappreciated  within  organizations  often  due 
to  the  lack  of  sponsorship  and  inadequate  attention  to  the  diffusion  of  the  new  technology.  During  the 
performance  analysis  phase  of  this  project,  many  non-training,  work  environment  factors  that  could 
contribute  to  the  success  or  failure  of  the  web-based  learning  environment  were  identified.  Critical 
diffusion  and  work  environment  factors  are  identified  below  to  highlight  some  of  the  biggest  challenges  to 
project  success  with  this  particular  target  audience.  Recommendations  relative  to  these  factors  have  also 
been  included. 


Redefining/Restructuring,  and  Clarifying  Relationships/Roles 

These  two  critical  stages  in  the  diffusion  of  a technology  within  an  organization  have  been 
documented  by  researchers  in  many  different  organizational  settings  (Rogers,  1995).  Following  initial 
design,  development  and  evaluation  of  the  web-based  learning  environment,  it  was  expected  that  members 
of  the  facilitator  community  will  begin  the  process  of  redefining/restructuringand  clarifying  rolesrelated  to 
this  technology.  Also  see  Communities  of  Practice  by  Wenger  ( 1998). 


The  following  recommendations  were  put  forth  in  an  effort  to  assist  in  the  initial  adoption,  and 
redefining/clarifying  of  the  prototype.  Long-term  evaluation  of  the  site  is  possible  only  after  adoption  and 
utilization  of  the  prototype  by  members  of  the  facilitator  community. 

Recommendations: 

• Identify  decision-makers  that  will  advocate  the  site.  This  is  the  identification  of  key 
stakeholders  within  the  facilitator  community  who  take  responsibility  for  the  product.  The  project 
sponsor  and  individual  training  units  will  be  responsible  for  identifying  key  adopters,  change  agents, 
and  problem-solvers  who  will  champion  the  use  of  this  innovation  by  members  of  the  facilitator 
community.  These  role  players  will  be  active  participants  in  the  design,  re-design,  and  evaluation  of 
the  application. 
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• Identify  implementation  managers  who  will  diffuse  the  site  throughout  the  facilitator 
community.  This  is  a critical  aspect  of  the  early  design  and  development  of  any  new  product.  These 
are  early  adopters  that  must  assume  the  burden  of  “selling”  the  product  to  others,  as  well  as  taking  on 
much  of  the  responsibility  for  its  success.  One  strategy  that  stakeholders  may  use  is  to  integrate  this 
tool  with  other  performance  support  tools  currently  in  use. 

• Introduce  the  site  during  ongoing  activities  already  existing  in  the  environment. 

- Introduce  the  site  during  initial  mentoring, 

- Have  trainers  and  mentors  introduce  the  site  as  a tool  for  individual  and  job 
development  The  site  could  help  experienced  facilitators  mentor  and  coach  less  experienced 
facilitators.  Facilitators  may  be  positively  influenced  to  use  the  site  if  it  is  encouraged  by  their 
mentors  as  a useful  way  to  develop  skills  and  build  new  knowledge. 

- Introduce  the  site  as  part  of  facilitator  training.  Introducing  the  site  as  a tool  that 
can  be  used  by  facilitators  to  build  knowledge  and  skills  related  to  job  duties  and  responsibilities 
will  increase  its  potential  impact  and  overall  effectiveness. 


Resources  for  ongoing  design  and  development 

Identification  of  adequate  resources,  human  and  financial,  to  support  the  maintenance  and 
continued  growth  ofthe  application  is  crucial  to  success.  This  is  an  often-neglected  aspect  of  computer- 
based  application  development,  especially  if  the  technology  requires  frequent  updating.  Web-based 
technologies  are  easier  to  update  than  other  technologies  and  thus  represent  a potential  long  term  cost 
savings. 

Recommendations: 

• Form  a team  responsible  for  learning  and  redefining  the  website.  Although  time  and 
monetary  requirements  are  minimal,  it  will  be  essential  to  identify  key  personnel  who  will  be 
responsible  for  supporting  ongoing  maintenance  of  the  site.  It  was  suggested  that  internal  facilitators 
who  are  familiar  with  the  facilitator  community  and  current  practices  be  trained  in  change  and  redesign 
processes.  A technical  support  resource  responsible  for  updating  and  refining  software  and  hardware 
will  also  be  required  for  continual  improvement  of  the  site. 

• Conduct  prototype  review  groups  with  potential  users  and  opinion  leaders.  Continual 
feedback  from  the  learning  community  is  helpful  in  redefining  and  redesigning  the  site  to  support  and 
build  on  current  course  content. 


Incentives 

Incentives  and  reward  systems  are  one  of  the  least  understood  yet  most  effective  methods  of 
improving  performance  in  the  workplace.  The  benefits  of  participating  in  the  learning  environment  include 
improved  performance,  self-development,  and  professional  growth. 

Recommendations: 

• Official  “release”  time  for  participation  in  learning  activities,  and  recognition  for 
participation  are  highly  recommended.  The  costs  of  release  time  can  be  considerable  but  the  potential 
benefits  of  participation,  in  terms  of  improved  performance,  are  well  worth  the  cost. 

• Facilitators  should  receive  positive  consequences  and  encouragement  from  their 
command  to  use  the  site  as  part  of  their  ongoing  skill  development. 

Process  Management 

Technical  and  human  resource  management  will  be  required  to  ensure  consistent,  supportive,  and 
performance-oriented  system  implementation.  The  interrelationship  of  this  new  process  with  other 
facilitator  processes  is  part  of  the  role  of  project  leaders. 

Recommendations: 

• Leaders  of  this  project  within  the  facilitator  community  must  be  fully  engaged  in  the 
process  of  creating,  refining,  communicating  and  using  knowledge  that  is  supported  by  technology. 


• Periodic  feedback  should  be  elicited  from  facilitators  to  continually  improve  and  add 
areas  of  interest  to  facilitators  that  are  actually  using  the  site. 


Equipment  (see  Site  Technical  Maintenance  section) 

Hardware  and  software,  physical  information  infrastructure,  and  documentation  will  be  required  to 
support  initial  and  ongoing  system  development. 


Recommendations: 

• Current  computer  hardware  and  software  must  meet  minimum  specifications  in  order  to 
capitalize  on  emerging  technologies.  This  is  probably  the  most  expensive  (monetarily)  of  all 
implementation  factors,  but  can  pay  off  in  learning  and  performance  effectiveness  in  the  longterm. 


Continuous  Improvement  of  the  Learning  and  Performance  Environment 

The  following  section  lists  the  major  components  of  the  web-mediated  learning  and  performance 
environment.  These  components  address  the  learning  objectives  required  to  achieve  selected  facilitator 
performance  requirements  or  best  practices.  By  addressing  the  best  practices  that  were  identified  through 
the  performance  analysis,  this  learning  environment  has  a greater  likelihood  of  having  a positive  impact  on 
facilitator,  performer,  and  organization  readiness.  The  pages  that  follow  offer  a brief  description  of  each 
component  of  the  learning  and  performance  environment,  and  recommendations  for  continuous 
improvement  and  maintenance  of  each  component  on  the  website.  Many  of  the  recommendations  included 
here  were  generated  during  the  participatory  design  processes. 


Learning  Environment  Component:  Practice 

Purpose : To  enhance  content  knowledge  and  problem  solving  ability  in  the  area  of  motivation 
through  a variety  of  Self-Assessment/  Content  Mastery  activities. 


These  activities  include  multiple  choice,  matching  and  “essay’Vcase  study  response  formats  in  an 
effort  to  enhance  situational  problem-  solving  ability. 


Recommendations  for  Ongoing  Development: 

- Monitor  and  update  self-assessment  items  to  reflect  current  course  content. 

- Continually  develop  new  items  that  challenge  users  and  reflect  current  research  and  practice 
in  the  area  of  motivation. 

- Ensure  relevant,  corrective  feedback  for  each  item. 

- Add  tutorials  that  provide  a review  of  classroom  material  for  each  level  of  the  course  guides 
for  motivation. 

- Identify  current  problems  to  which  facilitators  can  apply  basic  problem  solving  skills  to  solve 
larger  issues/concems. 

- Monitor  case  study  submissions  to  ensure  learning  opportunities  go  beyond  that  presented  in 
the  classroom. 

- Continually  add  new  case  studies.  'Expert'  facilitators  could  be  identified  to  update  and 
maintain  relevant  case  studies. 

- Review  submissions  by  peers  and  revise  as  necessary. 

- Revise  case  studies  as  field  experiences  change  oras  current  case  studies  become  outdated. 

- Hold  expert  forums  in  the  chat  room  and/  or  archive  interview  transcripts. 
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Learning  Environment  Component:  Discussion  Board 

Purpose : To  provide  current  learners  with  a platform  to  build  usable  knowledge  for  current  and 
future  facilitators  and  leaders  through  interaction. 

Features  include:  Message  Board  with  chat  feature  to  share  ideas,  experiences,  discussions,  etc., 
and  Synchronous  and  Asynchronous  communication  options. 

Recommendations  for  Ongoing  Development: 

Cleaning  up/Monitoring 

- Continually  monitor  information  for  accuracy. 

— Delete  outdated  or  incorrect  information. 

Review  other  discussion  boards  that  may  better  suit  user  requirements  (see  Site  Technical 
Maintenance  section). 

Maintaining  Interactions 

- Periodically  post  new  discussion  topics. 

- Encourage  participation  by  “graduates”  now  on  the  job. 

- Provide  positive  consequences  for  facilitators  to  interact  with  others  on  the  discussion 
board. 

- Reward  expert  facilitators  or  SME's  who  participate  in  scheduled  discussions. 


Learning  Environment  Component:  Help 

Purpose:  To  provide  assistance  and  reduce  frustration  in  navigating  through  the  site.  Features 
include  a Site  Map,  Technical  Help  with  FAQ’s  and  site  tips. 


Recommendations  for  Ongoing  Development: 

Updating/Revising 

As  facilitators  become  more  familiar  with  the  site,  Help  section  will  require  frequent 
review/revision. 

Monitor  and  delete/update  FAQ’s.  Facilitators  should  be  encouraged  to  ask  questions  of 
their  peers  and  to  respond  to  questions  appropriately. 

Revise  site  map/navigational  tips  based  on  revisions  and  advancements  to  the  site. 


Learning  Environment  Component:  Knowledge  Base 

Purpose : Where  all  other  features  converge  to  allow  for  storage  of  objective  (known  facts, 
theories,  procedures)  knowledge  and  to  promote  constructed  (new)  knowledge. 


Features  include:  Library  with  motivation-related  readings,  instructional  resources,  and  links  to 
other  organization  web  sites. 


Recommendations  for  Ongoing  Development: 

Monitor  and  update/ delete  “dead  links  ” to  remain  current . Many  online  resources  are  deleted  by 
the  site  authors  after  a certain  period  of  time  and  therefore  many  links  may  become  "dead"  links. 

Links  should  reflect  current  course  content  and  should  not  present  views  conflicting  with  current 
curriculum. 

Expert  contributions  should  be  archived  and  continuously  updated. 

Archived  models/theories  should  be  created  and  easily  accessed, 

A drop-down  menu  can  be  created  so  that  reactions  to  information  found  in  web  links  can  be 
accessed  under  each  link  in  the  Library, 
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Learning  Environment  Component:  Evaluation 

Purpose : To  provide  an  opportunity  for  the  user  to  aid  in  the  continual  development  of  the  site. 
Users  complete  a profile  that  identifies  them  by  location  and  level  experience,  among  other 
factors,  and  solicits  their  feedback  on  the  site.  This  profile  is  also  suitable  as  a Level  One  (Kirkpatrick) 
reaction  evaluation  tool. 


Recommendations  for  Ongoing  Development: 

Evaluation  responses  should  be  logged  and  continuously  monitored  to  assist  in 
usefulness  of  data  in  continuous  site  improvement . 

Survey  questions  (as  well  as  the  profile)  should  be  reviewed  based  on  changes  to  the  site 
and  evaluation  data  expected . 

As  updates  are  made  to  the  site,  face-to-face  evaluations  should  be  conducted  in  order  to 
observe  interactions  and  to  receive  feedback  from  facilitators  regarding  ease  of  navigation  and 
usefulness  of  activities . 


Site  Technical  Maintenance 

The  following  is  a brief  description  of  the  tools  and  applications  used  in  the  development  of  this 
project.  The  designers  provided  sufficient  information  for  an  Information  Technologist  to  understand  the 
basic  architecture  and  technical  features  of  this  product  for  the  purpose  of  continual  technical  development. 
Where  applicable,  possible  alternative  applications  to  the  ones  used  in  the  prototype  design  were  suggested. 


Platform : Windows  NT  4 running  on  Novell  Network  Server. 

■ Maintenance  Suggestions:  All  site  HTML  and  graphic  files  should  be  transferred  to  a Navy  server 
for  ease  and  control  of  ongoing  site  maintenance. 


Web  Interface : The  site  is  optimized  to  run  on  MS  Internet  Explorer  and/or  Netscape  Navigator 
browsers. 


Development  Applications : 

Microsoft  FrontPage  2000™ 

Macromedia  Fireworks  3™ 

Hypertext  Markup  Language  (HTML) 

Macromedia  Flash  4™ 

Notepad 

Microsoft  Access  2000™ 

Adobe  Photoshop  5.5™ 

Visual  Basic  Scripting™ 

Web  Crossing™ 

There  are  various  online  course  development  software  applications  that  have  also  gained 
popularity.  WebCT,  Blackboard  Courselnfo,  Phoenix  Pathlore,  and  Construe  are  just  a few  examples. 
Alternatively,  this  site  was  created  with  a variety  of  COTS  products  that  allowed  design  flexibility  for 
ultimate  user  customization. 
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Needs  Assessment  for  Design  and  Delivery  of 
Site-Based  Technology-Supported  School  Improvement  Degree 

Programs 

Jane  Zahner 

Valdosta  State  University 


Introduction 


With  schools'  increasing  accountability  for  student  achievement  comes  increasing  pressure  on 
teachers  and  administrators  to  know  how  to  effect  the  needed  changes.  This  paper  describes  the  needs 
assessment  activities  that  informed  the  design  and  development  of  M.Ed.  and  post-master's  degree  courses 
and  programs  to  be  delivered  onsite  and  through  distance  technology  support  in  rural  areas.  Two  partner 
school  systems  participated  in  the  needs  assessment  with  three  more  currently  involved  in  the  programs. 
Participating  school  systems  range  in  size  from  less  than  1000  K-12  enrollment  to  just  over  3000.  The 
University  partner  in  this  project  is  Valdosta  State  University,  a regional  university  in  South  Georgia  with 
an  enrollment  just  under  10,000.  VSU  has  a large  teacher  education  program,  enrolling  over  3000 
undergraduate  and  graduate  students  and  providing  training  for  a significant  percentage  of  the  educators  in 
the  largely  rural,  economically  limited  41  county  area  covering  1/3  of  the  geographic  area  of  Georgia. 

In  spring  2000  nearly  200  teachers  and  administrators  enrolled  in  a needs  assessment  course 
offered  by  the  Department  of  Curriculum  and  Instructional  Technology.  Some  of  these  educators  were 
already  involved  in  systematic  school  improvement  processes  at  their  schools,  others  were  not.  They 
enrolled  in  the  course  with  the  expectation  that  full  site-based  graduate  programs  were  in  the  design  and 
development  phase  and  would  incorporate  the  needs  assessment  course  as  a part  of  the  eventual  program  of 
study.  In  essence,  students  began  the  program  on  a promise  that  the  University  would  deliver  the  goods. 
What  those  goods  would  be  was  to  be  based  upon  the  results  of  needs  assessment  activities  taking  place 
concurrently  with  the  course.  Thus,  students  were  in  a unique  situation  of  simultaneously  studying  needs 
assessment  as  an  academic  topic,  conducting  needs  assessment  as  a part  of  their  own  school  improvement 
process  and  being  involved  as  subjects  in  a needs  assessment  conducted  by  the  author. 

In  fall  2000  the  promise  was  fulfilled.  Participants  began  to  earn  graduate  degrees  in  the  School 
Improvement  Degree  Programs  by  taking  innovative  courses  offered  onsite  at  their  rural  school  systems 
and  trough  distance  technologies.  They  are  enrolled  in  degree  programs  designed  and  developed  to 
specifically  improve  their  own  schools  and  systems,  while  increasing  their  own  professional  knowledge 
and  competence.  The  programs,  courses  and  modules  were  designed  based  on  the  extensive  school-based 
needs  assessment  conducted  during  the  spring  course.  Partnerships  among  school  personnel,  teacher 
educators  and  instructional  designers  have  led  to  programs  that  are  inclusive,  team  building,  action-oriented 
and  flexible.  Several  other  school  systems  in  the  region  have  asked  for  the  programs  to  be  phased  in  during 
the  next  two  years. 

Needs  assessment  methods 


This  paper  briefly  describes  the  needs  assessment  methods,  procedures  and  results.  Samples  and 
excerpts  from  the  needs  assessment  instruments  are  included  in  order  to  assist  others  who  are  doing  similar 
work.  While  one  direct  outcome  of  the  needs  assessment  was  an  initiation  of  a design  and  development 
competition  for  technology-delivered  course  modules,  that  process  is  described  only  briefly.  For  more 
information  about  the  courses,  modules  and  programs  contact  the  author  of  this  paper. 

A systematic  curricular  and  instructional  needs  assessment  was  conducted  at  two  rural  school 
systems  during  Spring  Semester  2000.  The  purpose  of  the  needs  assessment  was  to  gain  guiding 
programmatic  input  from  all  stakeholders.  This  input  was  essential  in  order  to  identify  the  educational 
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courses  and  programs  required  to  best  meet  the  educational  needs  of  the  individuals  enrolled  in  the  program 
and  to  achieve  the  educational  outcomes  of  the  school  systems  involved.  In  addition,  the  needs  assessment 
process,  data  and  resulting  report  established  the  framework  for  the  "charter"  degree  program  proposal 
submitted  to  the  University  System  of  Georgia  Board  of  Regents. 

Phase  One  of  the  needs  assessment  was  really  a 'wants  assessment'  (see  Phase  one:  Demographics 
and  wants  assessment).  Participants  (n=l  65)  were  first  asked  what  professional  development  / coursework 
would  be  most  useful  to  them  personally.  They  were  asked  to  include  the  title  and  content  as  well  as 
recommendations  for  delivery  method,  instructor,  and  other  participants.  This  survey  also  asked  for  name 
and  contact  information,  number  of  years  as  a teacher,  number  of  years  in  current  position,  subjects  taught, 
grade  levels  taught  and  leadership  roles,  if  any.  The  results  of  this  survey  were  compiled,  categorized  and 
distributed  to  the  participants.  This  served  as  a discussion  topic  for  groups  as  they  examined  their  personal 
needs/requests  in  the  context  of  school  improvement  needs  indicated  by  the  school  profiles  they  were 
building  during  the  course.  The  results  of  this  survey  indicated  a wide  variety  of  professional  development 
'wants'  with  teaching/managing  multiple  ability  levels,  time  management  and  technology  most  often 
mentioned. 

Phase  Two  of  the  needs  assessment  consisted  of  a survey  which  asked  teachers  to  identify  the 
grade-appropriate  knowledge/skills  their  students  lacked  at  the  beginning  of  the  school  year  (see  P hase  two: 
Student  readiness).  Analysis  of  these  data  resulted  in  a narrative  that  described  the  observed  skill 
deficiencies  across  P-12  educational  levels.  Reading  and  math  skill  deficiencies  were  observed  spanning 
the  entire  curriculum.  This  narrative  was  shared  with  the  participants  and  incorporated  into  their  data 
gathering  for  the  ongoing  School  Improvement  process. 

Participants  in  small  within-school  groups  held  peer-led  discussions  in  Phase  Three  of  the  needs 
assessment  (see  Phase  three:  Professional  development  needs  based  on  School  Improvement  Plan).  Groups 
were  asked  to  generate  five  content  areas  appropriate  for  School  Improvement  Degree  programs  with 
justification  based  on  documented  school  improvement  needs.  They  included  a short  description  of  the 
content,  target  audience  and  specified  delivery  options  from  a given  list.  Results  of  this  survey  generated 
the  following  broad  categories  of  professional  development/  coursework  have  been  identified  as  needed  to 
support  school  improvement  goals. 

School  Improvement  Areas  of  Need 


• Curriculum  Alignment 

• Differentiated  Instruction  (grouping,  at-risk,  etc.) 

• School-Business  Partnerships 

• Instructional  Strategies  (teaching  strategies,  direct 
instruction,  specific  programs,  etc.) 

• Analyzing  Test  Scores 

• Alternative  Assessment  (portfolios,  rubrics,  etc.) 

• Cultural  Diversity 

• Reading  Strategies 

• Writing  Strategies 

• Math  Strategies  and  Curriculum 


Motivation 
Test  Taking  Strategies 
Parental  Involvement 
Critical  Thinking  Skills 
Technology  Literacy 
Reference  Materials 
Legal  Issues 
Team  Management 
Interaction  With  Parents 
Time  Management 
Classroom  Management 


Phase  Four  asked  individuals  to  judge  and  report  competence  in  1 1 general  computer  skills  and  9 
computer  skills  dealing  with  Internet  use  (see  Phase  four:  Computer  ddlls/resources  checklist).  The 
Checklist  also  asked  participants  to  report  on  the  type  and  quality  of  computing  resources  available  to  them 
at  school  and  at  home  as  well  as  their  learning  preferences  in  regard  to  technological  delivery  of 
instruction.  Findings  from  analysis  of  the  data  produced  by  this  checklist  indicated  that  some  individuals 
would  need  extensive  basic  computer  instruction  in  order  to  participate  in  instruction  using  significant 
technological  delivery.  However,  findings  also  indicated  an  overall  fairly  high  level  of  computer  skills 
competence  and  a very  high  level  of  computer  access  including  Internet  access.  Participants  also  indicated 
a great  deal  of  interest  in  technological  delivery  of  program  instruction,  with  a concurrent  need  for  peer  and 
instructor  contact  as  well. 
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In  Phase  Five  of  the  needs  assessment  activity,  focus  groups  and  peer-led  discussion  groups 
involved  all  participants  in  multiple  assessments.  Participants  were  assigned  to  these  focus  groups  and 
discussion  groups  based  on  the  demographic  information  collected  in  Phase  One.  The  needs  assessment 
coordinator  facilitated  three  consecutive  focus  groups  (see  Phase  five:  Needs  assessment  coordinator-led 
focus  groups).  The  first  group  consisted  of  selected  counseling,  special  education,  social  work,  and  speech 
language  professionals  to  discuss  the  needs  for  content  from  their  areas  of  expertise  to  be  shared  with  the 
rest  of  the  school.  The  second  focus  group  brought  together  reading  teachers  to  generate  ideas  for 
extending  reading  instruction  across  all  courses/levels.  A third  focus  group,  composed  of  teachers  at 
’transition*  grades  from  elementary  to  middle  and  middle  to  secondary,  generated  ideas  for  the  improvement 
of  communication  between  school  levels.  Three  peer-led  discussion  groups  were  going  on  concurrently 
with  the  focus  groups.  The  first  (see  Phase  five:  Peer  discussion  group  A)  guided  participants  to  respond  to 
expert-suggested  'best  practices’  within  content  areas,  the  second  resulted  in  generation  of  ideas  about 
professional  development  needs  based  on  career  level  and  years  of  service  (see  Phase  five:  Peer  discussion 
group  B),  and  the  third  required  participants  to  create  a rubric  suitable  forjudging  educational  computer  use 
and  program  ideas  for  moving  teachers  to  higher  levels  of  computer  competency  (see  Phase  Five:  Peer 
discussion  group  C). 

Needs  assessment  instruments 

The  following  are  examples  of  either  entire  needs  assessment  instruments  or  excerpts  from  them. 
In  many  cases  the  format  of  the  instrument  tes  been  altered  to  save  space  in  this  paper.  For  more 
information  concerning  the  administration  of  these  instruments  or  the  analysis  of  the  data  collected  through 
their  administration,  please  contact  the  author  of  this  paper. 


Phase  one:  Demographics  and  "wants  assessment" 


Recall:  A need  is  a gap  between  what  is  and  what  ought  to  be.  In  order  to  design  the  School 
Improvement  program  and  courses  to  meet  your  individual  needs  and  the  needs  of  your  school,  your  input 
is  essential.  Tonight  and  during  the  next  few  weeks,  I will  be  asking  you  to  contribute  information  and 
ideas  about  what  will  make  this  program  work  for  you. 

I’ve  been  asked  to  design  a needs  assessment  that  includes  all  relevant  stakeholders  in  this  process. 
I'll  be  asking  you  to  complete  short  surveys;  I may  ask  that  some  of  you  participate  in  focus  groups  with 
me;  I may  request  short  telephone  interviews  with  some  of  you.  All  the  work  you’ve  been  doing  in 
collecting  information  for  your  school  profile  will  be  used,  as  will  your  work  on  the  school  mission  and 
vision.  I’ll  also  consult  with  administrators,  college  faculty  and  school  improvement  professionals  in  other 
programs.  I'll  look  at  the  professional  literature  to  see  what  successful  School  Improvement  degree 
programs  include  and  how  they  are  structured. 

Anyone  who  would  like  to  discuss  this  needs  assessment  or  to  contribute  ideas  for  the  program 

and  courses  is  welcome  to  contact  me  directly  at ore  -mail . Your  contributions  will  be  taken  seriously 

and  be  held  confidential. 

Please  complete  the  short  survey  below  and  turn  it  in  to  the  VSU  faculty  for  return  to  me.  Before 
you  turn  it  in,  please  tear  this  sheet  on  the  dotted  line.  I’d  like  for  you  to  have  this  needs  assessment 
description  to  keep,  as  well  as  the  record  of  my  telephone  number  and  e-mail  address. 


Name e-mail 


School work 

phone 

How  many  years  have  you  been  employed  as  a teacher? How  many  years  at  this 

school? 

Subject(s)  currently 

taught 
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Grade  level(s)  currently 
taught 

Do  you  currently  or  recently  act  as  a team  leader,  lead  teacher  or  have  other  similar  duties?  Briefly  explain. 


(On  reverse) 

Imagine  and  describe  a professional  development  course/activity  designed  just  for  you.  Title?  Content? 
Delivery  (in  class,  hands-on,  Internet,  etc.)?  Length?  Instructor  (generally,  who)?  Other  participants 
(generally,  who?)  What  need  would  it  serve? 

Phase  two:  Student  readiness 

Grade  level: (If  system  or  multiple  grade  levels,  please  indicate) 

In  a previous  survey,  some  of  you  indicated  that  you  are  faced  with  challenges  in  teaching  students  of 
multiple  abilities  in  your  classes.  This  survey  is  intended  to  ask  your  observation  of  areas  for  which  your 
students  may  need  more  and/or  different  instructional  preparation.  (If  you  don’t  teach  self-contained 
classes,  choose  any  observed  student  performance).  Write  about  at  least  three  examples  of  performance 
discrepancies  and  give  as  much  detail  as  possible. 

What  grade-level-appropriate  knowledge/skills  did  your  students  lack  at  the  beginning  of  this  school  year? 
Description  of  content?  Description  of  student  performance? 

Phase  three:  Professional  development  needs  based  on  School  Improvement  Plan 

This  survey  should  be  done  in  groups  of  10-15  with  a facilitator  leading,  organizing,  summarizing  & 
recording.  Each  group  should  hand  in  one  document  that  reflects  their  collaborative  work. 

• Generate  five  content  areas  for  coursework  appropriate  for  the  School  Improvement  Degree  programs, 
both  M.Ed.  and  Ed.S.  Be  sure  that  you  could  justify  the  content  based  on  documented  school 
improvement  needs. 

• Include  a short  description  of  the  target  audience  for  this  coursework,  if  not  appropriate  for  all 
participants. 

• Check  the  delivery  method  or  combination  of  methods  you  think  would  be  most  effective  for  the 
content. 

3 hr.  course  offered  over  entire  semester 

3 hr.  course  offered  in  1/2  semester 

Multiple-session  workshop 

Single-session  workshop 

Self-paced  module 

Peer- Supported  study  group 

Other 

Phase  four:  Computer  skills  / resources  checklist 

The  School  Improvement  Degree  Programs  Needs  Assessment  is  being  conducted  to  investigate 
course  and  module  delivery  possibilities  as  well  as  content  needs.  There  are  many  ways  instructional 
technology  could  assist  program  delivery  including  use  of  the  WWW,  CD-ROMs,  video,  e-mail  and  on-site 
hands-on  lab  courses.  This  checklist  is  to  gain  information  about  your  current  computer  skills  and  the 
computing  resources  you  have  readily  available  to  you.  NOTE:  No  decisions  have  been  made  to  use 
technology  in  delivery;  this  is  exploratory  only. 

Here  is  a list  of  general  computer  skills  that  would  be  useful  for  success  in  technology-integrated 
instruction.  Read  each  item  and  indicate  the  response  which  best  describes  your  current  ability. 


I can: 

• Start  up,  reboot,  and  shut  down  a computer 

• Start  and  quit  a program  stored  on  the  hard  drive 


Yes  No 
Yes  No 
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Save  and  retrieve  files  to  and  from  a floppy  or  the  hard  drive 

Cut/copy  text  from  one  source  and  paste  it  onto  another 

Open  and  close  menus  and  windows 

Move  and  resize  windows  on  your  desktop 

Navigate  a directory  structure  to  find  files 

Type  at  least  40  words  a minute 

Create  a word  processing  document 

Print  a word  processing  document 

Use  spell  and  grammar  checking  to  revise  my  work 

Log  onto  the  Internet 

Retrieve  and  delete  e-mail  messages 

Create,  send,  forward  and  reply  to  e-mail  messages 

Distinguish  between  an  e-mail  address  and  a web  address 

Send  group  mailings 

Post  messages  to  discussion  lists 

Locate  and  access  information  using  a WWW  search  engine 
Check  the  credibility  of  Internet  resources 

Locate  and  use  appropriate  computer  resources  and  technologies  within  a 
library  or  media  center 


This  section  asks  about  the  computer  resources  that  are  available  to  you  at  home  and  at  school 


Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

and  at  school 

2 1 .  Which  of  the  following  describes  your  home  computer?  (Check  as  many  as  apply) 

Do  not  have  home  computer  Computer  has  printer 

Computer  has  modem  Phone  line  available  to  computer 

Computer  has  CD  ROM  Drive  Connected  to  Internet  Servi  ce  Provider 

Computer  has  sound  card  & speakers  E-mail  account 


22.  Which  of  the  following  describes  your  classroom  computer?  (Check  as  many  as  apply) 

Do  not  have  classroom  computer  Computer  has  printer 

Computer  has  modem  Phone  line/network  line  available  to  computer 

Computer  has  CD  ROM  Drive  Connected  to  Internet  Service  Provider 

Computer  has  sound  card  & speakers  E-mail  account 


This  section  asks  about  your  instructional  preferences  in  the  area  of  course  or  module  delivery  via 
computer  technologies. 

23.  When  I am  asked  to  use  software  or  technologies  that  I haven’t  used  before  (such  as  e-mail,  VCR): 

a.  I look  forward  to  learning  new  skills. 

b.  I feel  apprehensive,  but  try  anyway. 

c.  I put  it  off  or  try  to  avoid  it. 

24.  If  I had  to  describe  my  predominant  learning  style/preference,  I would  say  it  is: 

a.  Auditory  - 1 leam  best  when  I can  listen  to  an  explanation  of  a concept. 

b.  Visual  - 1 leam  best  when  I can  read  the  course  materials  or  view  graphics  and  other  visuals. 

c.  Tactile  - 1 leam  best  by  “doing”. 

25.  Having  face-to-face  interaction  with  my  instructors  and  peers  is: 

a.  not  particularly  important  to  me. 

b.  somewhat  important  to  me. 

c.  very  important  to  me. 


What  are  your  ideas  about  how  technology  could  be  useful  in  delivery  of  the  School  Improvement  Degree 
Programs?  What  are  your  concerns? 

Phase  five:  Needs  assessment  coordinator-led  focus  groups 

In  this  activity,  held  simultaneously  with  other  Phase  5 activities,  participants  were  grouped  as  described 
below.  The  needs  assessment  coordinator  acted  as  leader  and  recorder  within  a 40  minute  time  period.  The 
following  questions  provided  the  framework  for  the  focus  groups. 

Focus  Group:  "Support  for  You" 

Participants:  8- 1 0 selected  counseling,  special  ed,  social  workers,  speech/language,  etc.  across  schools  and 
levels.  Groups  are  homogenous  by  support  function. 

Genera l Questions: 

1 . In  what  content  from  your  area  do  other  school  personnel  need  training? 

2.  Who  needs  training? 

3.  At  what  level(s)? 

4.  How  can  your  needs  be  addressed  in  a school  improve  ment  program? 

• Professional  development  needs? 

• Restructuring  of  time,  resources 

• New  curriculum  materials? 

Focus  Group:  "Reading  for  All" 

Participants:  8- 1 0 selected  reading  teachers  or  personnel  in  reading  support  areas  (media  specialists)  across 
schools  and  levels.  Groups  are  homogenous  by  function. 

General  Questions: 

1 . In  what  content  from  your  area  do  other  school  personnel  need  training? 

2.  At  what  level(s)? 

3.  Who? 

4.  How  can  your  needs  be  addressed  in  a school  improvement  program? 

• Professional  development  needs? 

• Restructuring  of  time,  resources 

• New  curriculum  materials? 

Focus  Group:  "Transitions" 

Participants:  8-10  selected  teachers  from  5th/6th  grades  or  8th/9th  grades.  Groups  are  homogenous  by 
teaching  assignment  in  a grade  which  transitions  from  elementary  school  to  middle  school  or  middle  school 
to  high  school. 

General  Questions: 

1 . How  can  your  needs  be  addressed  in  a school  improvement  program? 

• Professional  development  needs? 

• Restructuring  of  time,  resources 

• New  curriculum  materials? 

2.  How  can  communication  be  improved  between  teachers  at  different  schools? 

Phase  five:  Peer-led  discussion  group  activity  A 

In  this  activity,  held  simultaneously  with  other  Phase  5 activities,  participants  were  grouped  (maximum  of 
five  persons  per  group)  by  content  interest  and  grade  level  responsibilities.  Each  group  was  given  a list  of 
'promising  practices'  in  teaching  and  learning  in  their  areas  earlier  identified  by  experts  (teacher  educators 
and  arts  and  sciences  content  faculty).  They  were  asked  to  select  a leader  and  a recorder,  conduct  a 
discussion  and  construct  written  evidence  of  the  results  of  their  discussion  within  a 40-minute  time  period. 
The  following  questions  provided  the  framework  for  their  discussions. 
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Needs  Assessment  Summary:  Promising  Educational  Practices 

• What  methods/ideas  were  most  familiar  to  your  group  members?  Give  examples  of  ways  group 
members  have  used  familiar  methods/ideas. 

• What  methods/ideas  were  least  familiar  to  your  group  members? 

• Which  methods/ideas  are  applicable  across  educational  levels  (s)?  Which  are  not,  and  why? 

• How  did  you  rank  the  attached  methods/ideas  according  to  group  interest  in  further  study? 

Phase  five:  Peer-led  discussion  group  activity  B 

In  this  activity,  held  simultaneously  with  other  Phase  5 activities,  participants  were  grouped  (maximum  of 
five  persons  per  group)  by  years  of  experience  in  teaching.  They  were  asked  to  select  a leader  and  a 
recorder,  conduct  a discussion  and  construct  written  evidence  of  the  results  of  their  discussion  within  a 40- 
minute  time  period.  The  following  questions  provided  the  framework  for  their  discussions. 

Newcomer , Mid-Career  and  Seasoned  Veteran? 

• What  are  some  of  the  professional  development  activities  by  group  members  in  the  last  2 years? 

• What  did  the  group  decide  were  the  top  three  activities  and  why? 

• How  might  the  professional  development  needs  of  your  career-level  category  differ  from  those  of  other 
teachers? 

• How  might  the  professional  development  focus  of  your  school/system  need  to  change  to  facilitate 
school  improvement? 

Phase  five:  Peer-led  discussion  group  activity  C 

In  this  activity,  held  simultaneously  with  other  Phase  5 activities,  participants  were  grouped  (maximum  of 
five  persons  per  group)  by  convenience.  Each  group  was  given  the  following  activity  to  complete.  They 
were  asked  to  select  a leader  and  a recorder,  conduct  a discussion  and  construct  written  evidence  of  the 
results  of  their  discussion  within  a 40-minute  time  period. 

Build  a rubric 

A rubric  is  a way  of  describing  what  performance  ’looks  like'  at  various  levels  of  achievement.  For 
example,  the  following  might  be  a rubric  to  evaluate  the  "Performance  of  Automobile  Drivers". 


Level  1 : Novice 

Level  2:  Home  Town 

Level  3:  Standard 

Level  4:  Expert 

Has  driven  a few 
times  accompanied 
by  an  instructor.  Must 
be  reminded  to  signal, 
maintain  steady 
speed,  park  correctly 
and  yield  to  traffic.  Is 
worried  and 
uncomfortable  about 
having  to  drive. 

Drives  occasionally  in 
familiar  locations.  Often 
fails  to  signal,  maintain 
steady  speed,  park 
correctly  oryield  to  traffic. 
Is  uncomfortable  driving  at 
night,  in  rain  or  fog,  or  in 
moderate  traffic. 

Drives  regularly  and 
competently  in  generally 
familiar  locations  both  day 
and  night.  Seldom  fails  to 
signal,  maintain  steady 
speed,  park  correctly  or 
yield  to  traffic.  Is 
uncomfortable  driving  in 
adverse  conditions  including 
unfamiliar  locales, 
inclement  weather  and 
heavy  traffic. 

Drives  daily  with 
confidence  and  expertise. 
Always  signals,  maintains 
steady  speed,  parks 
correctly  and  yields  to 
traffic.  Can  easily  adapt  to 
driving  in  adverse 
conditions  including 
darkness,  unfamiliar 
locale,  inclement  weather 
and  heavy  traffic. 

Your  Task 

What  you  are  to  do,  as  a group,  is  to  build  a similar  rubric  to  describe  levels  of  expertise  in  using 
computers,  either  in  the  classroom  or  for  your  own  personal  productivity  at  school  or  at  home.  You  may 
want  to  discuss  computer  use  as  a whole  or,  even  better,  choose  a more  specific  topic  like  Internet  use, 
word-processing,  curriculum/technology  integration,  database  use,  etc.  Once  the  rubric  isbuilt  you  will  use 
it  to  look  at  your  own  skill  levels  and  generate  ideas  for  professional  development. 
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• Choose  your  topic. 


• Decide  on  and  name  your  levels  of  performance. 

• Discuss  what  a teacher  ’looks  like’  at  various  levels  of  performance. 

• Write  the  description  of  the  teacher  performance  at  the  various  levels. 

• Discuss  where  each  of  you  'fits'  within  the  rubric. 

• Mark  an  "X"  on  the  rubric  for  the  current  level  of  performance  for  each  group  member  (no  names). 

• Generate  ideas  for  what  kind(s)  of  courses/modules  offered  through  the  School  Improvement  Degree 
Programs  would  be  needed  to  move  teachers  to  higher  levels  of  competency  on  your  rubric. 

• Generate  ideas  for  what  kind(s)  of  resources  (hardware,  software,  facilities)  would  be  needed  to  move 
teachers  to  higher  levels  of  competency  on  your  rubric. 

Groups  build  rubric  on  supplied  form  and  answer  the  following  open-ended  questions. 

• What  kind(s)  of  courses/modules  offered  through  the  School  Improvement  Degree  Programs  would  be 
needed  to  move  teachers  to  higher  levels  of  competency  on  your  rubric? 

• What  kind(s)  of  resources  (hardware,  software,  facilities)  would  be  needed  to  move  teachers  to  higher 
levels  of  competency  on  your  rubric? 

Summary 

Program  and  course  development  for  the  School  Improvement  Programs  proceeded  based  on  the 
results  of  the  needs  assessment.  It  was  decided  that  the  courses  had  to  be  flexible  in  order  to  address  the 
documented  school  improvement  needs  of  the  teachers,  administrators,  resource  personnel  within  the 
schools  and  school  systems  involved.  To  meet  this  need  for  flexibility  some  courses  in  the  School 
Improvement  Degree  Programs  were  constructed  from  or  include  modules  of  varying  lengths.  Each 
program  participant  selects  modules  based  on  relevancy  to  his/her  teaching  and  the  needs  of  grade  level  or 
content  teams  or  groups.  In  order  to  assure  consistency  and  quality,  modules  were  developed  according  to  a 
template.  Instructors  were  encouraged  to  adapt  existing  modules  on  the  same  topic  building  knowledge  and 
skills  from  module  to  module.  Faculty  developing  modules  were  given  guidance  in  developing  modules 
within  the  template,  earned  a stipend,  and  were  supported  in  assembling  the  modules  for  duplication, 
storage  and  delivery.  Designer/developers  of  modules  were  required  to  teach  the  module  the  first  time 
through  and  revise  based  on  formative  evaluation  data.  They  may  or  may  not  be  assigned  to  teach  the 
module  at  later  points  in  the  school  improvement  program. 

The  combination  of  extensive  involvement  of  teachers  and  administrators  in  the  design  of  their 
professional  development  within  the  framework  of  student  achievement  has  powerful  potential.  Each  year 
school  systems  in  Georgia  write  consolidated  grant  applications  that  are  based  on  and  aligned  with  their 
school  improvement  plans  and  progress  indicators.  The  integration  of  this  process  with  the  School 
Improvement  Degree  Programs  has  broadened  participation  in  the  process  of  gathering  and  analyzing 
student  achievement  data  and  has  increased  interest  and  individual  responsibility  for  the  results. 

While  the  results  are  not  in  as  to  whether  these  School  Improvement  Degree  Programs  will  have 
an  impact  on  the  bottom  line  of  student  achievement  at  least  the  right  people  are  now  looking  for  that 
impact  — the  teachers  and  administrators  themselves  in  partnership  with  the  University. 
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Support  and  Training  for  High-School  Faculty 
Who  Will  Teach  Using  the  Web 


Zane  Olina 

Arizona  State  University 
Herb  Dwyer 

Tempe  Union  High  School  District 
Wilhelmina  Savenye 
Arizona  State  University 


Session  Objectives 

The  purpose  of  the  presentation  is  for  AECT  participants  to  learn: 

5.  About  the  major  barriers  that  teachers  face  when  trying  to  incorporate  web-supplemented  and  web- 
based  teaching  into  the  school  curriculum; 

• Practical  strategies  for  providing  adequate  support  and  training  to  teaching  staff  on  implementation  of 
web-supplemented  and  web-based  instruction  in  secondary  schools. 

Background  of  the  Virtual  School  Project 

Over  a six-year  period,  a team  of  staff  members  in  a high  school  in  the  Phoenix,  Arizona,  area 
explored  ways  to  improve  student  success  while  simultaneously  reducing  instructional  costs.  Technology  as 
a tool  for  change  played  a significant  role  in  most  of  the  solutions  that  were  explored.  Increasing  access  to 
computers  and  the  internet  by  students  both  at  school  and  off-campus  resulted  in  a proposal  for 
implementing  both  web-supplemented  and  web-based  instruction  on  a school-wide  basis. 

During  the  first  phase  of  the  project,  emphasis  was  on  piloting  two  classes/teachers.  One  teacher 
began  to  develop  web-supplemented  instruction  for  a freshman  English  class.  The  other  joined  the  faculty 
of  The  Virtual  High  School,  a consortium  project  administered  through  the  Concord  School  District  in 
Concord,  Massachusetts,  that  offers  web-based,  advanced  level  high-school  courses  to  students  from  all 
across  the  United  States.  It  is  envisioned  that  the  second  phase  of  the  project  will  expand  to  involve  all  high 
schools  in  the  district. 

From  the  start  of  the  project  it  has  been  clear  that  technology  is  simply  a tool  rather  than  a 
replacement  for  superior  instruction.  Teachers  have  been  and  will  be  the  primary  providers  of  good 
instruction.  Yet,  the  major  missing  component  in  most  technology-integration  projects  in  schools  has  been 
a lack  of  adequate  teacher  training  and  support  (Office  of  Technology  Assessment,  1 995;  President's 
Committee  of  Advisors  on  Science  and  Technology,  1997). 

The  Teacher-Support  Component 

The  focus  of  the  proposed  presentation  is  on  teacher  support.  Based  on  the  preliminary  work  of 
the  school-district  technology  personnel,  it  was  decided  to  conduct  a thorough  review  of  approaches  and 
strategies  for  teacher  professional  development  and  support  in  technology  integration.  It  was  envisioned 
that  this  study  would  further  inform  the  decision-making  process  regarding  the  implementation  of  web- 
based  and  web-supplemented  instruction  at  the  high  school  and  across  the  district.  The  investigation  was 
carried  out  as  a class  project  in  an  advanced  graduate-level  instructional -design  class.  The  class  project 
represents  the  beginning  of  a district-university  collaborative  effort.  This  presentation  will  describe  the 
major  findings  of  the  investigation  and  offer  practical  strategies  for  teacher  professional  development  and 
support. 
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Needs  Assessment  and  Content  Development 


During  the  teacher-support  investigation,  data  were  collected  from  four  main  sources.  First,  a 
review  of  relevant  print  and  online  resources  was  carried  out.  Secondly,  the  review  of  theoretical  and 
applied  articles  was  supplemented  by  analysis  of  individual  case  studies  of  professional -development 
programs  or  resources  available  for  teachers  on  the  web. 

Interviews  were  conducted  with  five  teachers  at  the  high  school.  These  teachers  were  identified 
by  the  campus  technology  coordinator  as  the  core  cluster  of  teachers  who  had  expressed  interest  in  using 
computers  in  the  classroom.  The  teachers  were  asked  about  their  concerns  regarding  the  use  of  computers 
in  teaching,  their  training  needs  and  the  types  of  support  that  they  would  need  for  implementing  web- based 
and  web-supplemented  teaching  in  the  classroom. 

Data  from  five  school  or  district-level  technology  coordinators  and  two  faculty  instructional - 
support  staff  members  were  also  collected  through  interviews  or  an  e-mail  questionnaire.  The  respondents 
were  asked  about  major  barriers  that  teachers  face  regarding  integration  of  computers,  training  needs  and 
best  practices  for  providing  professional  development  opportunities,  support,  and  incentive  systems  to 
teachers. 


A brief  summary  of  the  main  findings  is  presented  below . 

Major  Barriers  to  Technology  Integration 

The  most  common  barriers  to  technology  integration  in  teaching  are  the  increased  preparation 
time,  a lack  of  awareness  of  the  general  benefits  of  distance  education,  faculty  compensation  and 
incentives,  access  to  appropriate  technologies,  a lack  of  shared  vision  for  distance  education  in 
organization,  institutional  barriers  and  lack  of  support  staff  to  help  course  development  (Berge  & 
Muilenburg,  2000;  Moore  & Kearsley,  1996;  Office  of  Technology  Assessment,  1995). 

According  to  both  the  literature  sources  cited  above  and  the  accounts  of  the  teachers  and 
technology  coordinators,  the  lack  of  time  appears  to  be  the  single  major  factor  hindering  technology 
integration.  Robinson  (1995)  suggests  that  development  of  information  technology  in  education  can  be  seen 
as  part  of  the  broader  field  of  educational  change  in  which  there  is  a rich  and  useful  literature  (Fullan, 
1993).  Thus  technology-integration  efforts  should  be  addressed  as  part  of  systematic  efforts  at  improving 
classroom  practice . 

Teacher  Training  Needs 

The  University  of  Illinois  Faculty  Seminar  Report  (1 999)  highlights  two  distinct  features  of  online 
pedagogy.  First,  the  teaching  paradigm  must  change  from  the  traditional  lecture  format  to  one  more  suitable 
for  online  instruction.  Secondly,  the  instructor  has  an  important  role  in  moderating  the  interaction.  Porter 
(1997)  suggests  that  only  educators  possessing  certain  qualities  can  be  successful  distance-learning 
instructors.  Such  qualities  include  the  ability  to  learn  new  technology,  a performance  personality, 
flexibility,  and  time  to  create  new  materials  and  methods. 

Training  Models 

Teacher  training  should  focus  on  the  use  of  technology  in  teaching  rather  than  acquisition  of  skills 
using  software  (OTA,  1 995;  The  President’s  Committee  of  Advisors  on  Science  and  Technology,  1 997). 
No  single  approach  is  best  for  effective  teacher  professional  development  (OTA,  1 995)  and  a variety  of 
approaches  should  be  used  in  combination  at  any  given  time.  Some  common  strategies  are  developing 
technology  rich  classrooms  as  demonstration  sites,  training  master  teachers  who  then  serve  as  resources  to 
their  colleagues  and  providing  access  to  technical  support  staff. 


Findings 
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Support  Systems 

Support  systems  should  primarily  address  the  major  barriers  to  technology  integration  mentioned 
above.  Unless  a systemic  approach  to  technology  integration  is  adopted  individual  teacher  training 
initiatives  are  likely  to  be  ineffective.  Providing  time  for  experimenting  with  new  technologies,  and  the 
support  and  incentives  for  doing  so,  are  some  of  the  most  effective  strategies.  The  report  prepared  by 
University  of  Illinois  (1999)  emphasizes  the  need  for  recognizing  faculty  intellectual  property  rights  as  the 
best  way  of  assuring  high  quality  of  online  teaching. 

Implications  for  Teacher  Professional  Development  and  Support 

The  literature  review,  the  interviews  with  the  teachers,  and  the  conversations  with  the  technology 
coordinators  have  demonstrated  that  in  order  for  technology  integration  to  take  place  a comprehensive  and 
systemic  approach  is  necessary.  A number  of  practical  suggestions  for  implementing  teacher  training  and 
support  for  web-based  and  web-supplemented  teaching  follow.  The  suggestions  are  presented  in  three 
broad  categories:  setting  the  stage,  training  strategies  and  support  systems  for  sustaining  change. 

Setting  the  Stage 

Set  a Clear  Vision 

Have  a clear  vision  of  what  your  priorities  are  at  the  school  or  district  level.  Then  examine  ways  in 
which  technology  can  help  you  accomplish  those  broader  goals.  .Unless  teachers  see  the  link  between 
improving  quality  of  teaching  and  learning  and  use  of  technologies  in  the  classroom  any  technology 
integration  efforts  are  mots  likely  to  fail. 

Start  in  Small  Steps  That  Ensure  Success 

Start  in  small  steps  by  encouraging  teachers  to  initially  put  their  syllabus  and  class  assignments  on 
the  web,  or  to  create  brief  online  quizzes,  or  begin  using  electronic  grade  books.  Then  provide  them  with  a 
bigger  picture  of  what  the  next  steps  in  progression  might  be. 

Evaluate  Progress  Against  Initial  Goals 

Monitor  the  progress  and  measure  the  success  of  technology  integration  based  on  the  initial  goals. 
Technology  integration  is  not  about  the  number  of  computers  in  each  classroom  or  about  the  number  of 
internet  connections  it  is  about  a better  quality  of  teaching  and  learning. 

Involve  Faculty  in  Decision  Making 

Involve  faculty  in  formulating  the  vision  and  setting  priorities  for  your  school  as  much  as  possible. 
Hiring  external  instructional  designers  and  web  developers  may  seem  to  be  a quicker  and  more  efficient 
short-term  solution.  Yet,  this  approach  does  not  empower  teachers  to  try  out  un-conventional  instructional 
strategies  or  to  model  effective  technology  use  to  their  students. 

Establish  External  Partnerships 

Establish  external  partnerships  with  local  universities  and  businesses  for  additional  expertise  and 
support.  For  example,  graduate  students  from  universities  can  provide  instructional  design  expertise  to 
teachers  for  course  development.  Businesses  can  let  teachers  attend  their  training  events  or  provide 
volunteers  to  coach  teachers  on  technology  skills. 

Allocate  Adequate  Resources  for  Professional  Development 

US  Office  of  Technology  Assessment  recommends  that  at  least  30%  of  technology  funds  be  spent 
on  training.  Provide  teachers  with  release  time  for  jump-starting  new  web-based  courses,  or  stipends  for 
attending  training  workshops.  Compensate  teachers  instructing  their  colleagues  on  technology  skills  for 
planning  time  and  training  that  they  provide  after  school  hours. 

Provide  a Common  Set  of  Course  Development  and  Delivery  Tools 

Consider  adopting  a commonly  used  courseware  package  such  as  Web  CT,  Courselnfo  or  others 
for  the  entire  school  or  district.  P rovide  training  and  technical  support  for  the  users.  A common  courseware 
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package  provides  students  with  the  same  interface  for  all  the  courses  offered  and  they  are  not  distracted 
from  core  learning  tasks  having  to  master  a variety  of  applications.  A common  set  of  tools  requires  fewer 
resources  for  training,  support  and  management  of  web-based  instruction. 

Provide  Adequate  Infrastructure  and  Technical  Support 

Ensure  that  teachers  have  access  to  the  necessary  hardware  and  software  tools  for  developing  and 
implementing  web-based  and  web-supplemented  instruction.  One  of  the  models  that  has  proven  itself 
successful  has  been  providing  the  teacher  with  a laptop  and  equipping  the  classroom  with  a minimum  of 
five  workstations  with  internet  connection.  In  addition,  in  order  for  web-based  and  web-supplemented 
instruction  to  be  successful  students  also  need  to  have  access  to  computers  and  internet  outside  the  regular 
school  hours.  A high-speed  server  for  hosting  web-based  course  materials  is  also  an  absolute  necessity  o nee 
web -supplemented  and  web-based  instruction  becomes  more  widely  spread  at  the  school. 

Training  Approaches 

Pick  Your  Trainees  Carefully 

Web-based  instruction  requires  considerable  investment  in  training  and  daily  coaching.  Select 
your  trainees  carefully,  especially  at  the  initial  stages  of  adopting  web-based  instruction  at  your  school.  For 
the  trainees  to  be  successful  and  to  serve  as  role  models  for  their  colleagues  they  should  meet  a number  of 
criteria.  The  trainees  should  be: 

• Open-minded  and  willing  to  continually  learn  new  technologies  and  teaching  approaches; 

• Experienced  teachers  who  are  familiar  with  a range  of  teaching  strategies  in  face  to  face  situations  so 
that  they  have  a pool  of  ideas  to  pull  from  for  web-based  instruction; 

• Familiar  with  the  course  content  to  be  delivered  entirely  or  partially  over  the  web; 

• Comfortable  with  technology; 

• Capable  of  collaborating  with  others; 

• Willing  to  share  their  experiences  and  expertise  with  their  colleagues. 

Use  a Variety  of  Training  Approaches 

Provide  a wide  selection  of  training  opportunities  to  satisfy  the  varied  training  needs  of  the 
teachers.  Those  already  familiar  with  technology  will  need  only  additional  encouragement  and  ideas  for 
classroom  applications  of  technology  that  can  be  provided  over  the  web.  Novice  learners  will  need  very 
specific  initial  training  aiming  at  teaching  very  specific  technology  skills  and  their  applications  for  the 
classroom  and  ample  opportunities  to  practice  these  skills. 

Provide  Followup  Support 

Avoid  stand-alone  training  events  whenever  possible.  Instead,  offer  a series  of  training  events 
enabling  teachers  to  come  back  with  questions  and  to  try  out  the  newly  learned  skills  in  practice.  Provide 
follow-up  support  at  the  school  or  district  level  by  making  the  trainer  accessible  at  least  a few  times  a 
month  or  by  encouraging  the  participants  to  form  local  support  groups. 

Focus  on  Classroom  Application  of  Technologies 

Focus  all  training  on  the  classroom  applications  of  technology  rather  than  mastering  specific 
software  programs.  Emphasis  on  the  application  of  internet  or  software  programs  in  the  classroom  will 
provide  context  for  the  training  and  will  help  teachers  apply  the  newly  learned  skills  in  practice. 

Provide  Plenty  of  Real-Life  Examples 

Provide  teachers  with  a range  of  real-life  examples  and  case  studies  of  successful  technology 
integration  initiatives.  Whenever  possible,  the  examples  should  offer  solutions  to  the  challenges  and 
concerns  that  teachers  face  in  the  classroom  on  a daily  basis,  such  as  lack  of  student  motivation,  varied 
ability  and  interest  levels  of  a diverse  student  population,  lack  of  time  to  provide  immediate  feedback  on 
student  assignments  and  others. 
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Model  Technology  Use 

Model  technology  use  during  training  by  choosing  the  most  appropriate  delivery  medium.  If  you 
are  training  teachers  on  web-based  teaching  then  at  least  part  of  the  training  should  be  delivered  over  the 
world  wide  web. 

Provide  First  Hand  Experiences  With  Technologies 

Best  of  all  if  teachers  have  the  opportunity  to  participate  in  web-based  training  themselves  in  order 
to  step  into  the  students’  shoes.  If  that  is  not  possible  provide  as  much  hands-on  experiences  with  web- 
based  and  web-supplemented  instruction  during  your  training  as  possible. 

Just  in  Time  Training  Works  Best 

Timing  of  the  training  is  important.  Target  the  training  primarily  at  those  who  will  need  to  deliver 
web-supplemented  or  web-based  teaching  in  the  very  near  future.  Best  of  all,  design  the  training  around  the 
actual  course  projects  that  the  teachers  will  be  implementing  to  ensure  direct  application  of  the  newly 
learned  skills  in  practice. 

Allow  for  Plenty  of  Time 

Learning  takes  time  and  opportunities  for  trial  and  error.  Allow  trainees  time  to  try  out  smaller 
assignments  and  be  successful  before  launching  into  largersemester-long  course  design  projects.  Research 
shows  that  on  average  it  takes  around  three  to  five  years  for  teachers  to  become  comfortable  with 
technology. 

Do  Not  Have  the  Same  Expectations  for  Everyone 

Do  not  expect  every  faculty  member  to  integrate  technology  into  his  or  her  teaching  to  the  same 
extent.  Each  teacher  has  his  or  her  own  set  of  values  and  instructional  approaches  to  make  him  or  her  most 
effective  in  the  classroom.  Web-based  instruction  requires  a major  change  in  one’s  teaching  and  learning 
approach  and  the  medium  might  not  suit  everyone. 

Support  Systems  for  Sustaining  Change 

Provide  Ongoing  Technical  Support  for  Teachers  and  Students 

Once  web-supplemented  and  web-based  courses  are  offered  at  a relatively  large  scale  at  the  school 
or  at  the  district  both  teachers  and  students  must  have  access  to  a technology  help  desk  by  phone  twenty- 
four  hours  a day.  In  addition  to  campus  technology  coordinators,  school  librarians  or  media  specialists 
could  be  trained  to  provide  necessary  technical  support.  Librarians  could  also  provide  instructional  support 
to  teachers  by  locating  useful  web  sites  for  instruction  or  coaching  students  on  internet  search  skills  or 
online  study  strategies. 

Use  Students  in  Faculty  Support  Roles 

Ask  motivated  high  school  students  with  technology  skills  to  assist  faculty  in  putting  their  courses 
on  the  web.  Students  could  undertake  these  tasks  as  independent  study  projects  or  as  part  of  their  school  to 
work  experiences. 

Provide  Opportunities  for  Sharing  of  Ideas  and  Information 

Set  up  a curriculum  and  technology  resource  center  at  the  district  level  where  teachers  could 
access  to  additional  computers,  software  and  information  and  advice  regarding  web-based  instruction.  Set 
up  and  maintain  a support  web  site  for  teachers  where  they  could  access  online  tutorials,  share  lesson  plans 
and  plan  collaborative  course  projects. 

Consider  Adequate  Recognition  and  Rewards 

Recognize  teachers  who  excel  in  web-based  and  web-supplemented  teaching  as  instructional 
innovators.  Recognition  in  front  of  peers  is  one  of  the  most  effective  means  for  rewarding  an  accomplished 
teacher. 
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Develop  and  Implement  a Clear  Intellectual  Property  Rights  Policy 

Develop  and  implement  a clear  intellectual  property  rights  policy  at  the  school  or  district  level. 
Teachers  should  never  feel  that  by  developing  online  courses  they  will  some  day  lose  their  jobs  or  that 
someone  else  will  teach  their  courses.  Before  developing  new  online  materials  an  agreement  should  be 
signed  between  the  author  and  the  school  or  the  district  as  to  who  will  own  the  materials  once  it  is 
published  online  and  how  the  materials  will  be  used.  The  developer  should  always  be  asked  for  permission 
when  his  or  her  materials  are  to  be  used  for  other  courses. 
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The  Study 

The  site  for  this  study  was  the  west  suburban  Chicago  metropolitan  area.  The  school  district  was  a 
unit  district  providing  classes  for  students  in  grades  K- 1 2.  There  were  1 4 schools  in  the  district  with  a new 
elementary  school  and  new  high  school  under  construction  at  the  time  of  this  study.  The  high  school,  where 
this  research  actually  took  place,  had  an  enrollment  of  approximately  3200  freshman  to  senior  students. 

For  this  study,  28  participants  were  interviewed.  Of  these  participants,  13  were  teachers,  1 1 were 
students,  and  4 were  adult  support  personnel,  which  included  a library/media  specialist,  an  assistant 
principal,  a library  assistant,  and  a computer  technician.  Survey  data  were  also  collected  from  40  teachers 
who  had  access  to  the  media  distribution  system.  Teachers  reported  on  how  often  they  used  the  system, 
what  devices  they  used,  and  responded  to  evaluative  statements  on  using  the  system. 

The  transcripts  of  the  interviews  were  imported  into  a qualitative  data  analysis  computer  program. 
Interviews  were  then  coded  into  18  categories  for  retrieval  and  analysis.  The  information  that  emerged 
from  the  interviews  and  survey  was  divided  into  two  areas,  (a)  the  positive  impact  on  the  classroom  and  (b) 
the  areas  of  concern  that  participants  had  about  using  the  system. 


Positive  Impact 

Based  on  the  survey  and  interview  data,  the  following  summarizes  what  students  and  staff 

reported: 

1 . Classes  didn’t  have  to  worry  about  sending  and  receiving  carts  of  audio-visual  equipment  to  and 
from  classrooms.  The  ability  to  access  the  media  distribution  system  was  immediate. 

2.  The  system  was  easy  to  use  from  a staff  and  a student  perspective. 

3 . Class  time  was  interrupted  less  by  scrolling  the  daily  announcements  over  the  system  rather  than 
someone  reading  announcements  over  the  PA  system. 

4.  It  allowed  class  instruction  to  be  more  spontaneous.  The  “teachable  moment”  didn’t  have  to  get 
away,  the  class  now  had  immediate  access  to  a variety  of  multimedia  tools. 

5.  Teaching  styles  became  more  hands-off  while  student  participation  styles  became  more  hands-on. 

6.  Cable  TV  allowed  classes  to  view  world  events  and  late  breaking  news  as  it  was  happening. 

7.  The  media  distribution  system  helped  address  the  learning  styles  of  visual  learners.  Teachers  were 
able  to  present  material  in  a variety  of  ways. 

8.  A classroom  with  a media  distribution  system  was  more  efficient  than  a traditional  classroom. 
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9.  Classes  were  more  judicious  in  their  use  of  video.  Students  were  more  likely  to  view  snippets  of 

video  when  they  had  access  to  a media  distribution  system. 

10.  The  ability  to  network  classrooms,  to  view  media  in  multiple  classrooms  at  the  same  time,  was 

beneficial. 

1 1 . Teachers  reported  the  positive  effects  for  students  using  the  system  that  included  the  following: 

• Students  found  it  easier  to  present  electronically.  Students  were  able  to  move  from 
posterboard  speeches/presentations  to  multimedia  presentations. 

• Using  the  system  placed  the  attention  of  the  class  on  the  information  and  not  the  student. 

• Using  the  system  allowed  students  to  follow  their  interests  in  preparing  for  presentations. 

Survey  statements  number  1 and  16  related  to  the  overall  impact  on  the  classroom  environment. 

In  statement  one,  23  participants  (72%)  agreed  that  they  were  very  happy  with  the  media  distribution 
system  and  its  capabilities.  And  in  statement  number  16,  24  participants  (77%)  said  that  the  overall  impact 
of  the  system  on  the  classroom  environment  had  been  positive. 

Finally,  the  teachers’  comments  say  much  about  using  a media  distribution  system.  For  example, 
teacher  number  nine  stated  that  “it  has  changed  teaching  and  learning  in  our  class.”  Teacher  number  two 
voiced,  “It  makes  you  a better  teacher,  it  makes  your  class  a better  classroom  ” And,  teacher  number  seven 
stated: 


To  me  I guess  it's  kind  of  a continual  development.  It's  evolved  from  the  television  on  a cart 
thing,  to  this.  What  the  next  step  is,  I'm  not  sure.  But  the  point  is  that  this  is  a definite 
improvement  over  where  we  were. 

Areas  of  Concern 

Based  on  the  survey  and  interview  data,  the  following  summarizes  what  students  and  staff 

reported: 

1 .  Having  to  give  up  the  control  of  loading  the  videotapes  into  the  VCR  was  an  issue  expressed  by  both 

teachers  and  students.  Participants  did  not  like  the  fact  that  the  VCR  was  not  in  the  classroom  and  they 

gave  up  this  control  to  the  people  in  the  Resource  Center. 

2.  Teachers  said  that  the  27-inch  monitor  size  was  too  small.  No  student  reported  having  a problem 
with  the  monitor  size. 

3.  Sixty-two  percent  of  the  teachers  surveyed  said  they  don’t  really  need  all  of  the  bells  and  whistles 
included  in  a media  distribution  system.  They  just  need  a TV/VCR  in  their  room  and  it  would 
take  care  of  most  of  their  media  needs. 

4.  There  was  a strong  perception  that  the  media  distribution  system  is  just  for  showing  videotapes. 
Comments  like,  “Wouldn’t  it  be  cheaper  to  have  TVs  and  VCRs  in  each  classroom?”  reaffirms 
this  perception. 

5 . Only  1 2 teachers  (4 1 %)  thought  the  inservice  training  they  had  received  on  the  media  distribution 
system  was  appropriate  for  them  to  use  the  system. 

6.  Teachers  were  interested  in  additional  training,  19  teachers  (59%)  reported  they  were  interested  in 
more  inservice  training  on  the  media  distribution  system. 

7.  There  are  some  issues  in  running  and  maintaining  a media  distribution  system  including: 
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• Sensors  can  come  loose. 

• There  is  no  battery  indicator  in  the  remote  to  know  when  they  need  replacing. 

• Human  error  in  scheduling  or  loading  the  system. 

• CCM  box  at  the  top  of  the  monitor  can  be  disconnected. 

• Students  “play”  with  the  buttons  on  the  front  of  the  TV. 

8.  The  wireless  keyboard  used  to  access  the  computer  at  the  head  end  was  very  cumbersome  to  use. 

9.  There  is  a cost  of  maintaining  the  media  distribution  system  when  the  school  personnel  are  unable 
to  fix  hardware  or  software  problems.  An  outside  company  needs  to  come  in  to  complete  repairs. 

When  Buying 

Based  on  the  results  of  this  study,  there  are  several  important  points  that  a school  should  consider 
before  purchasing  a media  distribution  system.  These  would  include  the  following: 

• Plan  to  have  a computer  connected  to  each  classroom  monitor.  A media  distribution  system  is  a 
presentation  system  for  both  students  and  teachers.  A computer  will  enhance  the  potential  of  the 
system  by:  (a)  giving  the  teacher  the  ability  to  schedule  media  from  the  classroom  and  (b)  the  ability  to 
show  computer  images  to  the  class.  Purchase  the  computer  with  a TV  card.  This  will  allow  the  user  to 
connect  to  the  TV  with  just  an  RCA  cable. 

1 . Purchase  the  largest  monitor  that  is  practical  for  the  budget.  At  the  site  that  this  study  was 
conducted,  27-inch  monitors  were  installed.  Some  of  the  teachers  thought  that  this  size  did  not 
provide  adequate  viewing  for  the  entire  room.  A larger  monitor  will  provide  a larger  viewing  area 
for  the  classroom.  A school  should  also  have  the  option  of  connecting  an  LCD  projector  to  the 
system.  An  LCD  projection  system  in  the  classroom  will  allow  for  a significantly  larger  viewing 
area. 

2.  Consider  the  monitors  that  accept  a computer  signal.  Traditionally,  to  show  a computer  image  on 
a TV  screen,  a scan  converter  was  needed  to  convert  the  computer  signal  to  a signal  that  the  TV 
monitor  could  “understand”  or  show.  Some  monitors  will  accept  the  computer  signal  and  provide 
much  better  resolution  when  showing  computer  images.  Students  and  teachers  will  appreciate  the 
better  resolution. 

3 . Purchase  monitors  without  buttons  on  the  front  of  it.  Or,  some  monitors  let  the  user  internally 
disable  the  buttons.  This  will  be  less  of  a distraction  for  students  who  want  to  explore  what  each 
button  does. 

4.  One  of  the  negatives  that  came  out  in  the  study  was  that  teachers  did  not  like  the  idea  of  giving  up 
control  of  the  videotapes  to  the  personnel  operating  the  system.  A school  could  consider 
providing  VCRs  with  some  of  the  monitors,  especially  if  a teacher  uses  many  videotapes  during  a 
specific  class.  The  price  of  VCRs  have  come  down  to  a point  that  would  make  this  idea  more 
possible. 

5.  Inservicing  the  teachers  is  a key  to  the  successful  operation  of  the  media  distribution  system. 
Although  the  system  is  easy  to  use,  the  more  that  teachers  know  of  how  to  use  the  system,  the 
greater  the  potential  for  teaching  and  learning.  Plan  on  initial  training  and  also  follow-up  training 
once  the  teachers  have  acclimated  to  the  system. 

Written  Survey  Data 

The  following  are  the  teacher  responses  from  the  anonymous  survey. 

This  study  can  be  viewed  on  the  World  Wide  Web  at:  http.V/www. inil.com/users/bobmm/dissuse.htm 
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Media  Devices  Used 

Daily 

Weekly 

Monthly 

Never 

Total  Responses 

CDI  Player-Encyclopedia 

0 

0% 

0 

0% 

4 

13% 

28 

88% 

32 

CDI  Player-  Audio  CD's 

0 

0% 

0 

0% 

5 

16% 

27 

84% 

32 

AM/FM  Tuner 

0 

0% 

3 

9% 

2 

6% 

27 

84% 

32 

VCR 

2 

6% 

14 

44% 

16 

50% 

0 

0% 

32 

Laser  Disc  Player 

0 

0% 

1 

3% 

4 

13% 

27 

84% 

32 

Still  Video  Floppy  Player 

0 

0% 

2 

6% 

7 

22% 

23 

72% 

32 

Computer  to  Monitor 

0 

0% 

1 

3% 

5 

16% 

26 

81% 

32 

Cable  Channels 

Daily 

Weekly 

Monthly 

Never 

Total  Responses 

History  Channel 

1 

3% 

0 

0% 

0 

0% 

31 

97% 

32 

CBS 

1 

3% 

0 

0% 

4 

13% 

27 

84% 

32 

NBC 

1 

3% 

0 

0% 

5 

16% 

26 

81% 

32 

WTTW 

0 

0% 

0 

0% 

6 

19% 

26 

81% 

32 

WCIU 

1 

3% 

0 

0% 

1 

3% 

30 

94% 

32 

CNN 

0 

0% 

2 

6% 

9 

28% 

21 

66% 

32 

Weather  Channel 

0 

0% 

1 

. 3% 

6 

19% 

25 

78% 

32 

Arts  & Entertainment 

1 

3% 

1 

3% 

1 

3% 

29 

91% 

32 

CNN  Headline  News 

1 

3% 

1 

3% 

6 

19% 

24 

75% 

32 

Discovery  Channel 

1 

3% 

0 

0% 

3 

9% 

28 

88% 

32 

Survey  Statement  Responses 

SA&  A 

No  Opinion 

D&SD 

Total  Responses 

1)  Very  happy  w/capabilities 

23 

72% 

4 

13% 

5 

16% 

32 

2)  Inservice  appropriate 

12 

41% 

7 

24% 

10 

34% 

29 

3)  Rather  teach  in  an  MDS  room 

22 

69% 

8 

25% 

2 

6% 

32 

4)  27-inch  monitor  is  large  enough 

16 

52% 

1 

3% 

14 

45% 

31 

5)  Would  like  more  inservice 

19 

59% 

6 

19% 

7 

22% 

32 

6)  Import,  use  engage  learning 

26 

87% 

4 

13% 

0 

0% 

30 

7)  MDS  helps  engage  students 

22 

69% 

9 

28% 

1 

3% 

32 

8)  More  efficient  w/MDS  classroom 

19 

59% 

9 

28% 

4 

13% 

32 

9)  Prefer  just  TVA/CR 

9 

28% 

11 

34% 

12 

38% 

32 

10)  MDS  provides  rich  media  environ 

21 

68% 

10 

32% 

0 

0% 

31 

1 1)  Believe  in  rich  M/environ 

28 

88% 

3 

9% 

1 

3% 

32 

12)  Prefer  traditional  classroom 

4 

13% 

9 

29% 

18 

58% 

31 

13)  Efficient  w/lessons  using  media 

14 

44% 

9 

28% 

9 

28% 

32 

14)  Use  more  media 

13 

41% 

8 

25% 

11 

34% 

32 

15)  21st  Century  classroom 

24 

75% 

6 

19% 

2 

6% 

32 

16)  Overall  impact  positive 

24 

77% 

6 

19% 

1 

3% 

31 
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Design,  Development  and  Evaluation  of  a Virtual  Histology  Course 


Regina  Schoenfeld-Tache 
Sherry  McConnell 
Colorado  State  University 


Abstract 

This  paper  describes  the  design  and  development  of  a distance -delivered,  upper-level  science 
course,  and  the  results  of  a study  conducted  to  determine  its  effectiveness  relative  to  the  on- 
campus  section  of  the  same  course . This  course  takes  advantage  of  the  unique  characteristics  of 
the  Internet  by  using  it  to  transmit  visual  information  and  allow  for  asynchronous  and 
synchronous  interactions . The  assessment  conducted focused  on  content  mastery  and  the  effect  of 
computer-mediated  communication  on  type  and  depth  of  questions.  No  significant  differences  in 
academic  achievement  were  observed  be^een  the  two  groups.  However ; the  rate  at  which 
students  and  faculty  interacted  was  much  greater  in  the  online  sessions  (chat  and  review),  and  a 
greater  percentage  of  higher  order  questions  were  asked  in  the  online  sessions. 

Introduction 

WebCT  Inc.  recently  recognized  online  Histology  as  an  exemplary  course  due  to  its  academic 
rigor  and  content  robustness.  This  paper  will  briefly  outline  the  design  and  development  processes  involved 
in  creating  the  course  and  subsequently  focus  discussion  on  the  results  of  a study  comparing  virtual 
histology  to  a traditional  (on-campus)  section  of  the  same  course  taught  simultaneously  by  the  same 
instructor. 

Course  Design  and  Development 

A needs  analysis  to  determine  the  feasibility  and  value  of  offering  an  upper  level  science  course 
via  the  Internet  was  conducted  by  one  of  the  authors  (SMC).  She  discovered  a need  for  upper  division 
science  courses  offered  in  an  asynchronous  format  in  order  to  meet  the  needs  of  students,  mainly  those  of 
non-traditional  ages  and  backgrounds,  pursuing  pre-health  majors.  Scheduling  conflicts  made  it  particularly 
difficult  for  students  with  family  and  work  responsibilities  to  attend  traditional  on-campus  science  lectures 
and  laboratories.  Having  taught  Histology,  a study  of  tissues,  for  many  years  in  the  on-campus 
environment,  she  was  able  to  recognize  its  potential  for  conversion  to  online  delivery.  Given  the  nature  of 
the  subject,  the  course  is  image-intensive,  making  it  ideal  for  Internet  delivery.  WebCT  was  selected  as  the 
delivery  system  because  of  its  ability  to  offer  controlled  access  and  rapid  downloading  of  images  via  its 
CDROM  tool.  The  goal  was  to  create  an  online  course  that  would  preserve  the  same  objectives  as  the  on- 
campus  course  while  harnessing  the  unique  properties  of  the  delivery  system  to  allow  for  other  types  of 
instructional  activities.  The  resulting  course  emphasizes  the  concepts  of  histology  rather  than  the 
development  of  psychomotor  skills  relative  to  physical  manipulation  of  a microscope. 

Although  at  the  time  of  the  study  all  required  components  (including  examinations)  were  offered 
asynchronously  (participation  in  chat  sessions  was  optional),  in  its  current  form,  the  online  histology  course 
offers  asynchronous  lectures  and  laboratories  and  synchronous  examinations  and  online  discussions  (chats). 
An  expanded  syllabus  (a  combination  of  a calendar,  traditional  syllabus  and  learning  objectives)  guides 
students  and  permits  self-pacing.  Lectures  are  designed  to  simulate  real  lectures  instead  of  being  a modified 
textbook.  Clinical  examples  and  everyday  analogies  emphasizing  content  applications  are  frequently 
inserted  in  the  lectures.  Students  are  guided  by  objectives  and  a text  reading  assignment  for  each  lecture.  At 
the  end  of  each  lecture,  a humorous  slide  of  the  instructor  leads  students  to  the  formative  quizzes 
(“bonehead  quizzes”),  which  provide  them  with  instant  feedback  about  their  level  of  learning.  These 
quizzes  are  not  graded  because  they  are  designed  to  scaffold  metacognitive  skill  development  and  add  some 
humor  to  the  lectures.  Each  week  at  a scheduled  time,  the  students  are  invited  to  participate  in  an  online 
discussion  of  current  course  topics.  Chat  sessions  are  well  received  by  the  students.  The  instructor  leads 
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these  sessions  with  questions  about  content.  In  addition,  students  are  called  upon  to  ask  each  other  specific 
questions  about  the  material.  The  result  is  the  creation  of  a sense  of  community  (there  are  often  humorous 
asides  and  social  interactions),  where  members  support  each  other  and  value  individual  contributions  to  a 
group  learning  environment.  During  these  sessions,  students  are  confirming  their  level  of  knowledge  and 
gaining  insights  about  ::how  the  instructor  asks  questions.’'  On  their  own,  students  have  started  setting  up 
additional  online  discussions  each  week. 

Online  Histology  addresses  a variety  of  learning  outcomes,  ranging  from  simple  declarative 
knowledge  to  the  development  of  higher-order  cognitive  skills,  such  as  scientific  problem  solving.  The 
manner  in  which  the  characteristics  of  the  Internet  were  harnessed  to  address  each  type  of  outcome: 

• Content  knowledge:  The  distance  class  was  a section  of  the  traditional  course.  For  the  sections  taught 
by  SMC,  both  the  online  and  on-campus  courses  covered  the  same  material,  had  the  same  objectives, 
and  had  the  same  examination  questions.  The  only  difference  between  the  online  and  face-to-face 
sections  was  the  manner  in  which  the  material  was  presented.  While  most  of  the  face-to-face 
instruction  took  the  form  of  a didactic  lecture,  the  bulk  of  online  instruction  was  presented  through 
visually  stimulating  web  pages  containing  practical  examples  that  allowed  the  learners  to  control  the 
pace  of  instruction. 

• Problem  solving  skills:  Online  science  courses  have  been  criticized  for  lack  of  laboratory  experiences 
(Carr,  2000).  In  the  online  histology  course,  this  problem  has  been  overcome  by  the  creation  of  an 
interactive  online  1 aboratory  that  provides  an  experience  very  similar  to  that  of  an  on-campus  histology 
laboratory.  Using  the  Internet,  a series  of  images  captured  at  various  powers  of  the  microscope  are 
sequenced  and  linked  together  to  simulate  the  views  from  a microscope  at  different  powers.  The  only 
difference  between  the  online  traditional  laboratories  is  that  students  are  not  physically  manipulating  a 
glass  slide  and  focusing  a microscope  in  the  online  version  of  the  course.  Since  these  psychomotor 
outcomes  may  be  of  value  to  students  pursuing  careers  in  histology /pathology,  an  on-campus  lab  is 
offered  as  a supplement.  However,  the  lack  of  psychomotor  learning  is  more  than  compensated  for  by 
the  increased  amount  of  questions  discussed  in  class  and  the  greater  depth  of  thought  (and  interaction 
with  the  content)  demonstrated  by  students  in  the  online  class. 

• Metacognitive  skills:  Learning  science  at  a distance  requires  students  to  acquire  higher  order  cognitive 
skills  and  metacognitive  skills,  such  as  reflective  analysis,  independent  thought,  time  management  and 
self-motivation.  Frequent  e-mail  and  online  chat  interactions  with  the  instructor  provide  support  for 
students  while  encouraging  them  to  take  responsibility  for  their  own  learning. 

As  part  of  the  development  process  for  this  course,  differences  and  similarities  of  online  and  on- 
campus  environments  were  considered.  There  was  a conscious  attempt  to  create  an  inviting  but  challenging 
online  environment  that  would  parallel  the  academic  rigor  expectations  of  the  traditional  course.  Almost 
5,000  images  were  collected  and  inserted  into  the  course  in  order  to  fully  present  the  content  in  an 
interactive  environment.  Administrative  and  colleague  resistance  was  encountered  throughout  the 
development  process.  Concerns  of  academic  rigor,  “seat  time,”  appropriate  generation  of  official  credits, 
and  how  to  assess  faculty  teaching  in  this  medium  for  purposes  of  promotion  and  tenure  were  and  still  are 
encountered  frequently  in  the  development  and  implementation  of  this  course.  Finally,  the  instructor  has 
always  enjoyed  interacting  with  students  in  her  on-campus  courses,  and  was  concerned  the  opportunities 
for  this  type  of  contact  might  be  diminished  with  the  online  course.  The  instructor  was  able  to  confirm  that 
instructor-student  interactions  occurred  with  at  least  as  much  frequency  in  online  environments  as  in  the 
on-campus  settings. 

Course  Evaluation 

Recent  trends  (International  Data  Corporation,  2000)  in  distance  education  make  it  necessary  to 
explore  the  effects  of  online  delivery  on  students’  academic  and  personal  growth.  Since  an  ever-increasing 
number  of  courses  are  being  offered  at  a distance  it  is  important  to  investigate  the  quantitative  and 
qualitative  effects  of  an  online  delivery  medium  on  student  outcomes. 

The  Histology  course  gave  us  a unique  opportunity  to  extend  the  current  research  on  distance 
education  into  the  realm  of  advanced  science  courses-  a topic  of  great  concern  in  the  current  literature.  We 
were  able  to  investigate  the  effectiveness  of  the  Internet  as  a tool  to  teach  science  at  a distance,  as  well  as 
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how  the  medium  affected  interaction  among  participants  (students  and  faculty)  thanks  to  the  availability  of 
parallel  course  sections  taught  by  the  same  instructor. 

Study  Design 

The  research  questions  addressed  in  the  study  were: 

• Is  online  delivery  as  effective  as  face-to-face  instruction  for  science  classes?  Specifically,  was  there 
any  difference  in  academic  outcomes  between  the  sections? 

• How  does  computer-mediated  communication  affect  interactions  in  an  upper-level  science  class? 
Specifically,  does  it  affect  the  number  or  type  (depth)  of  questions  asked? 

• If  there  are  any  differences  in  interactions,  how  do  these  changes  affect  learning  outcomes? 

Participants  in  this  study  were  students  enrolled  in  either  the  on-campus  or  distance  section  of 
Histology  at  Colorado  State  University.  Forty-four  students  volunteered  to  take  part  in  the  study.  Of  these 
students,  33  were  enrolled  in  the  on-campus  section  and  1 1 took  part  in  the  online  section.  Student 
participation  was  observed  in  three  settings:  face-to-face  lecture  (traditional  instruction),  online  class 
(instructor-led  synchronous  discussion)  and  online  review  (student  initiated  study  session,  without 
instructor’s  presence). 

In  order  to  answer  the  aforementioned  research  questions,  students  in  both  sections  were  asked  to 
complete  a diagnostic  quiz.  This  quiz  provided  a baseline  measurement  of  all  students’  prior  knowledge. 
The  academic  performance  of  students  in  the  on-campus  section  was  measured  by  their  performance  on 
relevant  sections  of  a lecture  exam  (material  taught  by  the  participating  faculty  member).  In-class 
interactions  (questions  asked)  for  this  group  were  recorded  by  direct  observation.  Students  in  the  online 
section  allowed  the  researchers  to  access  their  exam  results  electronically.  The  conferencing  software 
automatically  recorded  online  interactions  during  chat  sessions.  In  addition,  students  in  the  online  section 
completed  a course  survey  addressing  their  perceptions  of  this  type  of  instruction. 


There  was  no  statistically  significant  difference  ( =0.05)  between  the  experimental  and  control 
groups  on  either  the  pre-  or  post-test.  However,  the  amount  of  interactions  (questions  asked)  varied  greatly 
by  setting.  During  face-to-face  lecture  sessions,  an  average  of  1 8 interactions  took  place  per  hour.  Online 
class  sessions  averaged  51  interactions  per  hour,  and  online  review  sessions  had  an  average  of  54 
interactions  per  hour.  Students  assumed  a more  active  role  in  the  online  sessions  than  they  did  during  the 
face-to-face  lectures,  despite  being  specifically  asked  to  participate  in  the  lecture  sessions.  Students 
initiated  4 1 % of  questions  during  online  classes,  but  only  34%  during  face-to-face  lectures.  When  the  topic 
of  these  questions  was  examined,  it  was  determined  that  the  questions  asked  during  lecture  sessions  were 
almost  exclusively  focused  on  content  topics  (87%),  and  no  social  questions  were  observed  in  this  setting. 
A student  spontaneously  assumed  the  role  of  facilitator  during  the  review  sessions,  which  led  to  a 
comparable  amount  of  management  interactions  in  both  online  settings  (27-28%).  The  main  difference 
between  online  class  and  review  sessions  was  the  shift  of  10%  of  the  interactions  from  content  topics 
during  class  sessions  to  social  interactions  during  the  review  sessions.  An  average  of  66%  of  interactions 
were  dedicated  to  content  topics  during  the  online  class  sessions,  but  only  56%  during  online  review 
sessions.  A more  in-depth  examination  of  the  content  interactions  revealed  that  a substantial  amount  of 
interactions  (60%)  asked  during  lecture  sessions  were  of  a low  cognitive  level  (levels  1 and  2 of  Bloom’s 
Taxonomy  - knowledge  and  comprehension).  Both  the  instructor  and  students  asked  a larger  proportion  of 
high-level  questions  (lewis  5 and  6-  synthesis  and  evaluation)  during  the  online  class  sessions  than  they 
did  during  the  face-to-face  lecture  sessions.  The  vast  majority  of  interactions  in  review  sessions  were  at  the 
lower  levels  of  Bloom’s  Taxonomy 


The  instructor  was  able  to  overcome  obstacles  common  to  faculty-development  of  online  courses  by 
persistence  and  creativity.  Her  experiences  can  serve  as  an  example  of  how  faculty  can  develop  exemplary 
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online  courses  regardless  of  level  of  computer  skills.  Ongoing  feedback  from  students,  and  instructional 
design  staff  have  resulted  in  ongoing  modifications  to  best  suit  the  needs  of  the  students  and  the  medium. 

It  is  interesting  to  note  that  even  though  the  instructor’s  presence  had  only  a minor  effect  on  the 
number  and  topic  of  interactions  in  each  setting,  her  presence  did  cause  a major  shift  in  the  level  of  thought 
reflected  in  the  questions  asked  in  each  setting.  Although  the  online  review  sessions  had  a higher  frequency 
of  interactions,  these  interactions  consisted  primarily  of  low-level  questions  which  could  be  answered 
rapidly,  both  in  terms  of  the  amount  of  time  required  to  think  about  the  content  and  formulate  a response  as 
well  as  in  the  time  required  to  physically  type  an  answer. 

The  “anonymity”  afforded  by  computer-mediated  communication  encouraged  more  in-depth 
thought  and  processing  in  student  answers  during  the  chat  sessions.  Students  were  more  likely  to 
participate,  had  more  time  to  reflect  and  ask  thoughtful  questions  compared  to  the  lecture  settings. 

The  main  conclusion  of  this  study  was  that  it  is  possible  to  teach  upper  level  science  courses 
without  sacrificing  academic  rigor.  When  face-to-face  and  online  sections  of  the  same  course  were 
compared,  academic  outcomes  were  the  same,  but  the  online  classes  demonstrated  a deeper  engagement 
with  the  content,  and  had  a qualitatively  different  experience. 
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Abstract 


Ease  of  use  is  critical  to  promoting  success  in  computer-assisted  instruction.  The  Rookie  Camp  unit  is 
aimed  at  preparing  high-school  students  for  the  WebCT  working  environment  and  provides 
explanation  on  how  to  be  successful  using  WebCT.  The  unit  was  developed  to  introduce  high-school 
students  to  the  use  of  the  icons  and  related  course  management  tools  on  the  " default " WebCT  course 
homepage.  This  paper  includes  descriptions  of  the  design,  development  and formative  evaluation  of  the 
Rookie  Camp  unit.  Implications  for  the  future  are  based  on  the  outcomes  of the  formative  evaluation  as 
well  as  findings  from  the  literature  review  focusing  on  student  support  in  computer-  and  web-assisted 
instruction. 

Introduction  and  Project  Background 

Improving  student  success  while  reducing  instructional  costs  can  be  a tremendous  challenge  for 
many  higher-education  institutions.  It  is  interesting  to  note,  however,  that  many  high  schools  across  the 
country  are  facing  a similar  challenge. 

At  one  high  school  in  the  southwest,  two  staff  members  at  the  school  teamed  up  to  search  for 
solutions  to  meet  this  as  well  as  other  academic  challenges.  Over  a six-year  period,  they  explored  many 
scenarios,  reviewed  the  literature,  and  participated  in  model  projects.  As  a result  of  their  efforts,  they 
concluded  that  the  use  of  technology  in  the  classroom  could  possibly  provide  some  solutions. 

With  the  recent  passing  of  a bond  issue  by  the  voters  in  the  school  district,  funding  for  necessary 
capital  equipment  was  secured.  Among  other  things,  the  bond  issue  provided  for  installation  of  a high- 
bandwidth,  high-speed  infrastructure  with  access  to  the  Internet  during  the  1998-99  school  year.  With  the 
foundation  in  place,  the  challenge  was  on  how  to  successfully  utilize  this  system  to  meet  the  high  schools' 
academic  goals. 

The  team  developed  a proposal  outlining  ways  to  increase  student  success  rates  and  decrease  costs 
by  using  technology  in  the  classroom.  Some  of  the  solutions  proposed  included  providing  web- 
supplemented  additional  learning  activities  and  opportunities  for  students  identified  as  “turnaround 
students”,  those  students  that  are  at  risk  of  failing  required  courses.  Other  solutions  offered  opportunities 
for  advanced  students  to  take  college  level  courses  while  in  high  school.  Finally,  web-based  instruction 
could  help  those  students  who  may  not  be  able  to  be  on  campus  for  health,  disciplinary,  or  other  reasons. 

Since  it  was  unrealistic  to  expect  teachers  to  become  web  developers  immediately,  the  first 
project  initiated  was  the  development  of  a web-supplemented,  rather  than  fhlly  web-based,  Freshman 
English  class.  The  decision  to  begin  with  a web-supplemented  course  would  allow  for  materials  to  be 
developed  gradually,  tested,  evaluated,  and  revised  (Dwyer  & Boyle,  1999).  Development  of  the  course 
materials  began  in  the  spring  of  1999  and  the  materials  are  being  used  at  the  present  time. 

Efforts  to  Expand  the  Use  of  Web-Supplemented  Activities 

The  high  school  chose  WebCT  to  be  the  courseware  used  for  web-supplemented  instruction.  In  an 
effort  to  increase  the  use  of  technology  in  the  classroom,  members  of  an  Arizona  State  University  graduate 


class,  Advanced  Instructional  Development,  were  brought  in  to  assist.  During  the  semester,  we  spent  time 
at  the  high  school  speaking  with  the  school  administrators,  media  director,  teachers  and  students.  Based  on 
our  meetings,  we  outlined  several  projects  that  would  be  of  value. 

Focus  on  the  Learner 

Introducing  and  actually  being  successful  in  implementation  of  web-based  courses  is  an  arduous 
task.  Many  obstacles  can  present  themselves  quickly  at  the  mention  of  utilizing  the  web  as  a tool  for 
instruction.  The  difficulties  may  lie  not  only  in  gaining  agreement  with  administration  and  faculty,  but 
concern  has  to  be  given  to  the  students  who  will  be  trying  to  increase  their  learning  in  this  new 
environment. 

Though  it  is  probably  true  that  many  high-school  students  have  some  computer  experience, 
relatively  few  have  had  exposure  to  using  the  web  as  part  of  their  classroom  instruction.  In  order  to  insure 
success,  the  learners  need  to  be  ready  to  work  with  this  new  tool.  Learner  readiness  involves  gaining 
competencies  in  using  navigation  tools  and  becoming  familiar  with  the  learning  environment  (Twigg,  1 999; 
Winiecki,  1999). 

In  order  to  ease  the  transition  to  web-supplemented  courses  and  promote  success,  the  Rookie 
Camp  unit  was  developed.  This  unit  will  become  a “default  tool”  to  be  included  in  all  WebCT  courses  at 
this  high  school.  Rookie  Camp  will  be  the  first  unit  the  high-school  students  in  web- supplemented  courses 
will  go  through. 

Description  of  the  Rookie  Camp  Unit 

This  unit  is  centered  on  a school  or  "Camp”  environment,  including  graphics  of  students  taking 
notes  and  a cheerleader  leading  the  Rookie  Camp  Cheer.  The  unit  is  designed  to  make  learning  both 
effective  and  enjoyable. 

Rookie  Camp  is  a self-instructional  unit  in  WebCT.  Directions  are  provided  to  the  student 
throughout  the  unit,  along  with  reminders  of  the  goals  of  the  unit.  The  student  is  also  provided  a Rookie 
Camp  Reminder  Card.  The  Rookie  Camp  Reminder  Card  is  a single  sheet  with  a graphic  of  each  of  the 
nine  "default"  WebCT  course  homepage  icons.  The  student  is  encouraged  to  write  notes  next  to  each  icon 
on  the  Reminder  Card  as  he/she  is  working  through  the  unit. 

Rookie  Camp  Instructional  Flow 

The  unit  begins  with  a screen  displaying  the  Rookie  Camp  title  and  asking  the  student  to  click  on 
the  "Course  Materials"  icon  to  enter  Rookie  Camp.  (Figure  1)  After  doing  this,  the  student  is  provided 
another  screen  with  directions  to  click  on  the  "Course  Content"  icon  to  begin  using  the  Rookie  Camp  unit. 
This  is  the  same  manner  in  which  the  student  will  access  all  of  their  other  courses  in  WebCT. 
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The  student  is  then  presented  a Welcome  screen  with  a description  of  Rookie  Camp  unit,  followed 
by  a screen,  which  shows  a list  of  Rookie  Camp  Activities.  The  activities  are  an  overview  of  the  areas  they 
will  be  working  through  in  the  unit.  The  student  is  then  provided  the  actual  learning  objectives  of  the  unit; 
after  completing  the  Rookie  Camp  unit  they  will  be  able  to  provide  the  name  of  each  of  the  nine  "default" 
WebCT  course  homepage  icons,  and  describe  the  tools  the  icons  represent. 

Since  the  student  is  not  familiar  with  WebCT,  WebCT  and  the  "default"  course  homepage  are 
explained.  Next,  the  student  is  introduced  to  the  Rookie  Camp  cheer:  McClintock  Chargers  Can  Sure 
Leam  Cool  Stuff  As  Required.  The  first  letters  of  each  word  in  the  cheer  represents  the  name  of  one  of  the 
nine  WebCT  icons.  The  student  will  be  reminded  of  the  cheer  later  in  the  program  and  told  to  use  it  as  a 
learning  tool,  as  it  is  a mnemonic  device. 

The  student  is  then  presented  nine  screens,  each  with  a graphic  of  one  of  the  icons  and  a detailed 
description  of  the  uses  of  the  icon.  The  descriptions  provide  information  on  how  the  student  will  use  the 
tools  represented  by  the  icon  for  various  course  management  functions.  This  section  focuses  on  the  actual 
learning  objectives.  A sample  of  an  icon  screen  is  shown  in  Figure  2. 


Calendar  Icon 


The  Calendar  icon  represents  tools  that  provide  a quick 
and  easy  way  for  you  to  find  out  about  important  dates  in 
your  course. 

You  will  use  this  set  of  tools  to  find  out  when  assignments 
are  due,  or  when  a test  is  scheduled. 


The  calendar  tools  will  help  keep  you  organized  and  on 
track. 


Figure  2.  Sample  Icon  Screen 

After  working  through  the  Rookie  Camp  sections  concentrating  on  the  names  and  tools 
represented  by  each  of  the  nine  WebCT  icons,  the  student  is  then  directed  to  work  through  the  practice 
section  of  the  unit.  The  student  is  again  reminded  that  he/she  is  responsible  for  knowing  the  name  of  each 
icon  and  the  tools  each  icon  represents.  The  practice  section  consists  of  1 8 multiple-choice  type  questions. 
For  all  of  the  questions,  a graphic  of  one  of  the  nine  icons  is  shown.  In  nine  questions,  the  student  is  asked 
to  select  the  name  of  the  icon  from  the  responses  provided,  and  in  nine  more  questions  the  student  is  asked 
to  select  the  tools  represented  by  the  icon  from  the  responses  provided. 

When  student  has  completed  the  practice  section,  he/she  is  told  that  the  Rookie  Camp  Cheer  can 
help  in  remembering  the  names  of  the  icon.  The  student  is  then  asked  to  come  up  with  their  own  strategy 
for  remembering  the  tools  the  icon  represents.  This  is  the  review  section  of  the  unit.  (In  the  final  version 
of  the  unit,  the  student  may  be  reminded  that  he/she  can  go  back  through  the  unit  to  review  any  of  the  icons 
they  may  have  had  difficulty  learning.  The  student  can  go  back  to  any  section  by  clicking  on  the  "pg  back" 
button  or  by  clicking  on  the  hyperlink  of  the  section  displayed  on  the  left  side  of  the  WebCT  screen.  This 
information  is  not  provided  in  the  prototype  unit.  Due  to  time  constraints  in  the  formative  evaluation  or 
try-out  process,  it  was  best  for  students  to  work  through  the  unit  in  a linear  fashion.) 


The  unit  is  then  concluded.  The  student  is  then  directed  to  complete  the  unit  quiz  and  attitude 
survey.  For  purposes  of  the  formative  evaluation,  a paper-based  quiz  was  used.  In  order  to  program  a quiz 
in  the  WebCT  format,  it  is  necessary  to  "attach"  the  quiz  to  information  in  a student  database.  This 
information  was  not  available  during  the  development  of  the  unit. 

Formative  Evaluation  of  the  Unit 

Procedure 


During  the  Spring  2000  semester,  a formative  evaluation  of  the  Rookie  Camp  unit  took  place  at 
the  high  school.  The  formative  evaluation  consisted  of  having  students  work  through  the  entire  unit  and 
complete  the  unit  assessment  and  attitude  survey.  The  try-out  was  conducted  in  a Freshman  English 
class.  The  teacher  was  beginning  to  develop  activities  in  WebCT  for  her  class,  so  was  very  interested  in 
conducting  the  try-out  with  her  students.  The  classroom  was  equipped  with  six  computers  with  Internet 
access,  so  the  teacher  selected  six  students  to  participate  in  the  tryout.  Of  the  six  students,  three  were 
considered  "high-achievers"  and  three  were  considered  "lo\wachievers".  All  students  were  ninth 
graders. 

When  I arrived  at  the  classroom  I accessed  the  Rookie  Camp  unit  from  each  of  the  six  computers. 
There  were  three  computers  on  one  side  of  the  room  and  three  on  the  other  side  of  the  room.  The  teacher 
was  conducting  class  while  I was  readying  the  computers.  Once  the  computers  were  ready,  each  of  the 
students  sat  down  at  a computer.  I gave  them  each  a Rookie  Camp  User  Guide  and  Rookie  Camp 
Reminder  Card.  They  were  asked  to  read  the  User  Guide  and  begin  working.  The  Rookie  Camp  User 
Guide  stated  that  Rookie  Camp  is  a unit  of  instruction  focusing  on  the  nine  "default"  WebCT  course 
homepage  icons.  In  Rookie  Camp  they  would  be  introduced  to  the  names  of  each  icon  and  the  tools  each 
icon  represents.  Since  the  unit  covers  a lot  of  material,  it  was  suggested  they  take  notes  on  the  Rookie 
Camp  Reminder  Card.  They  were  to  work  through  the  unit  in  its  entirety  making  sure  to  read  each  screen 
carefully. 

Unit  Assessment  Results 

The  unit  assessment  was  a paper-based  instrument,  as  indicated  earlier.  There  were  nine  sections 
with  each  section  showing  a graphic  of  one  of  the  nine  WebCT  homepage  icons.  In  the  space  next  to  the 
icon,  the  students  were  directed  to  write  in  the  name  of  the  icon,  and  describe  the  tools  the  icon  represents. 

Overall,  students  did  well  on  the  assessment.  For  all  nine  icons,  all  six  students  were  able  to 
provide  the  correct  names  of  the  icons,  the  first  learning  goal.  Describing  the  tools  the  icon  represents  was 
slightly  more  challenging.  The  student’s  scores  for  this  learning  goal  averaged  67%  correct.  In  fact,  one 
student  did  not  provide  a description  for  any  of  the  tools  of  the  nine  icons;  it  is  not  clear  if  they  did  not 
understand  the  directions,  did  not  remember  any  tool  descriptions,  or  ran  out  of  time  - the  change  of  class 
bell  rang  while  some  students  were  still  completing  the  assessment. 

Attitude  Survey  Results 

The  attitude  survey  consisted  of  1 0 4-item  Likert  type  statements,  and  two  open-ended  questions. 
Summary  of  the  Likert  statements  indicated  that  all  six  students  agreed  that  the  unit  directions  were  clear, 
the  unit  was  about  the  right  length,  they  enjoyed  the  unit  and  would  recommend  it  to  their  classmates. 

The  first  open-ended  question  asked  the  students  what  they  really  liked  about  the  Rookie  Camp 
unit.  Student  responses  included  the  directions  being  clear  and  the  unit  being  simple  and  easy  to  use. 

The  second  open-ended  question  asked  students  what  would  improw  the  Rookie  Camp  unit.  The 
students  suggested  using  more  pictures  and  color.  This  is  very  interesting  since  there  are  pictures  and/or 
color  on  all  screens.  One  student  indicated  that  there  was  too  much  content,  while  one  student  even  stated 
that  the  unit  was  perfect! 

In  general,  it  appears  that  the  students  enjoyed  working  through  the  unit.  For  the  most  part,  they 
were  successful  at  learning  the  names  of  the  icons,  but  still  need  additional  instruction  and  practice  in  order 
to  learn  the  tools  represented  by  the  icons.  A revision  to  the  unit  should  possibly  include  providing  the 
student  with  practice  activities  using  the  icons  and  related  management  tools  to  complete  various  tasks. 
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Implications  for  Learner  Support 

The  Rookie  Camp  unit  in  WebCT  provides  students  with  an  introduction  to  the  elements  with 
which  they  will  be  working  in  a web-supplemented  learning  environment.  Ease  of  use  is  critical  to 
promoting  success  in  computer-assisted  instruction.  If  the  students  continually  need  help  in  overcoming 
obstacles  in  the  software,  they  will  become  frustrated  quickly  (Heinich,  Molenda,  Russell,  & Smaldino, 

1 998).  Based  on  these  concerns,  students  using  web-based  instruction  should  become  familiar  with  the 
navigational  requirements  of  the  system  being  used,  as  well  as  suggested  study  skills  to  enhance  their 
success. 

Additional  Implications 

In  addition  to  becoming  familiar  with  the  computer  as  a new  learning  tool,  student  success  is  also 
predicated  on  the  use  of  successful  study  skills  and  learning  strategies.  Students  participating  in  distance 
education  courses  have  stated  that  improving  study  skills  is  a key  element  (Visser  & Visser,  2000).  The 
skills  that  students  need  in  order  to  succeed  in  all  forms  of  instruction,  especially  individual  instruction, 
play  an  important  role  in  effective  learning  in  computer-assisted  instruction.  Some  of  these  skills  include 
planning  and  organizing  for  learning,  and  learning  to  apply  skills  developed  in  the  classroom  or  paper-and- 
pencil  environment  to  complete  computer-assisted  instruction  (Cates,  1991). 
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Abstract 


This  paper  presents  strategies  and  rationales  for  implementing  certain  instructional  techniques  to 
move  a class  from  cohort  to  community.  The  context  is  the  new  Distance  Master’s  program  in 
Instructional  Systems  Technology  at  Indiana  University.  The  authors  give  suggestions  for 
instructional  and  non-instructional  strategies  that  have  students  interacting  at  the  levels  of 
discussion,  cooperation  and  collaboration.  These  strategies  are  cross-indicated  with  their  intended 
outcomes,  that  is,  strengthening  the  feeling  of  community  as  defined  by  a set  of  characteristics, 
which  are  adapted  from  Schwier  (in  press).  Suggestions  for  evaluation  techniques  are  also 
presented,  as  are  questions  for  further  research. 

Introduction 

The  shift  from  traditional  classroom  education  to  computer-mediated  distance  learning  poses 
enormous  challenges  to  instructors  and  learners.  The  concept  of  the  classroom  where  students  meet  to 
interact  with  other  learners  and  the  instructor  no  longer  exists.  The  instructor  can  no  longer  “look”  around 
the  room  to  see  if  students  are  attending  to  the  material,  bored  or  confused.  Learners  lack  a natural  social 
outlet  to  engage  with  other  learners  thus  leading  to  feelings  of  isolation.  The  learner  is  now  engaged  with 
the  computer  instead  of  other  learners.  The  big  question  for  our  project  is  “How  do  we  structure  the  course 
design  so  learners  have  mechanisms  to  connect  with  each  other  and  form  community.”  How  do  we 
overcome  the  characteristics  of  the  medium  so  that  learners  feel  connected  to  the  instructor  and  other 
learners? 

The  literature  on  effective  teaching  and  learning  promotes  several  “big  ideas”  that  we  used  as 
foundations  for  our  recommendations.  These  include  Vygotsky’s  ( 1 978)  social  development  theory  and  the 
Seven  Principles  for  Good  Practice  in  Undergraduate  Education  (Chickering  & Gamson,  1 987).  Vygotsky’s 
social  development  theory  states  that  social  interaction  is  vital  to  cognitive  development;  all  higher-order 
functions  originate  as  the  relationships  among  individuals.  To  scaffold  learning  we  must  require  learners  to 
interact  with  the  content,  the  teacher  and  each  other.  Our  strategies  focus  on  promoting  communication, 
social  interaction  and  participation.  Many  of  the  principles,  theories  and  strategies  we  encountered  reflect 
the  Seven  Principles  of  Good  Practice  in  Undergraduate  education  (Chickering  & Gamson,  1 987).  At  their 
core,  each  of  the  seven  principles  focuses  on  interaction.  In  1 996,  Moore  and  Kearsley  described  three 
types  of  interactions  that  are  necessary  in  distance  education:  learner  to  learner,  leaner  to  content  and 
learner  to  instructor.  We  would  argue  that  these  three  types  of  interaction  are  necessary  in  education 
regardless  of  where  or  how  it  takes  place. 
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Characteristics  of  Community 

There  is  much  discussion  of  learning  communities,  communities  of  practices,  and  virtual  or  online 
(social)  communities.  Although  each  type  of  community  has  its  distinct  characteristics  and  requirements, 
there  are  many  things  they  share  in  common.  What  we  are  endeavoring  to  create  will  be  a combination  of 
all  of  the  aforementioned  communities:  a community  of  practice  (since  our  cohort  will  be  from  the  same 
company)  that  is  involved  in  mutual  learning  online.  Because  of  these  special  characteristics,  some  things 
do  not  apply.  For  example,  there  is  much  talk  in  the  virtual  community  literature  about  attracting  members 
and  defining  the  community  based  on  common  interests.  In  our  case,  this  cohort  is  thrown  together  and 
“forced”  to  form  community.  Outside  members  are  not  encouraged  to  participate,  mainly  because  the 
common  interest  in  this  case  is  “taking  the  Distance  Masters  in  1ST  from  IUB.”  In  a terrestrial  community 
of  practice,  members  might  see  each  other  at  work,  or  meet  in  person  once  a week  to  deal  with  issues  in 
their  work  lives.  This  will  not  exactly  be  the  case  for  our  community;  although  they  will  probably  have 
some  work  issues  in  common,  they  are  not  a group  of  “teachers”  or  “nurses”  or  “engineers”  who  share  vast 
amounts  of  experience  and  knowledge.  Unlike  an  informal  learning  community,  which  spins  itself  from 
nothing  and  is  based  on  a variety  of  people  coming  together  for  informal  learning  purposes  and  where  the 
direction  of  both  the  learning  and  the  community  is  malleable,  our  learning  community  will  exist  within 
strict  parameters  of  this  coursework.  Obviously,  members  will  be  encouraged  to  bring  other  experiences 
and  knowledge  to  bear  on  their  coursework,  but  at  the  end  of  the  day,  the  learning  in  question  will  be  much 
more  restricted  than  an  informal  learning  community. 

Selznik  (1996)  identifies  seven  elements  of  community:  history >,  identity,  mutuality,  plurality, 
autonomy,  participation,  and  integration.  With  respect  to  virtual  learning  communities,  Schwier(in  press) 
adds:  an  orientation  to  the  future , technology,  and  learning.  Some  of  these  characteristics  of  community 
will  be  present  from  the  beginning.  Others,  the  cohort  will  have  to  grow  into.  We  will  describe  the  features 
of  these  1 0 characteristics,  and  discuss  how  we  will  use  them  for  our  purposes. 

Selznik  notes  that  communities  are  stronger  when  their  members  share  history  and  culture,  rather 
than  simply  abstract  general  interests.  Unlike  an  established  terrestrial  community,  the  nascent  community 
forming  from  the  distance  education  masters  program  will  not  have  a shared  history . Their  history,  like 
their  identity,  will  have  to  grow  and  develop  through  their  interactions  with  each  other. 

We  believe  that  a shared  sense  of  identity  will  develop  in  this  cohort,  and  will  strengthen  their  communal 
identity.  Schwier’s  suggested  strategies  for  fostering  identity  include  team-building  exercises,  developing 
community  logos,  and  public  acknowledgement  of  individual  and  group  accomplishments  within  the 
community.  He  also  notes  the  importance  of  articulating  the  “focus  or  purpose  of  the  community”  and 
outlining  the  requirements  and  rituals.  The  structure  of  the  courses  allows  for  frequent  and  obvious 
reiteration  of  community  focus,  and  events  such  as  orientation  can  help  the  group  define  its  own  rituals  and 
norms. 

The  very  fundamentals  of  a learning  community  require  interdependence  and  reciprocity,  what 
Selznik  terms  mutuality.  Since  our  focus  is  on  cooperative  and  collaborative  learning,  this  mutuality  will 
develop  naturally.  Schwier  also  recommends  asking  “leading  questions  that  encourage  members  of  the 
community  to  invest  in  concerns  held  by  other  members,  and  to  share  ideas  and  possible  solutions”  (p.  5). 
This  type  of  interaction  can  be  encouraged  at  course-level  in  the  class  forum,  and  on  a social  level  in  the 
Online  Cafe. 

We  combined  Selznik’s  terms  history,  mutuality,  and  identity  into  a larger  category  called  group 
identity.  By  combining  these  three  concepts  we  emphasize  the  fundamental  importance  of  group  identity  in 
fostering  community.  Although  one  of  our  goals  in  the  next  few  semesters  is  to  help  students  begin  to 
construct  a history  relevant  to  their  community,  this  is  not  something  that  can  be  imposed  upon  the  group 
from  outside.  It  has  to  grow  from  the  sharing  of  each  individual’s  history  and  the  links  that  the  learners 
form  with  each  other  based  on  their  experiences.  These  links  are  characterized  by  interdependence  and 
reciprocity,  in  other  words  mutuality.  Group  identity  results  from  this  history  and  mutuality,  and  from 
making  the  budding  community  history  public  and  available  to  all,  especially  newcomers. 

Plurality \ according  to  Selznik,  results  when  many  different  types  of  interactions  amongst 
members  of  a community  occur,  often  rooted  by  individuals’  membership  in  other  communities  (work, 
neighborhood,  church,  etc.)  that  intersect.  We  replaced  plurality  with  social  interaction.  Given  a virtual 
community,  one  that  to  some  extent  is  externally  imposed,  the  opportunities  for  plurality  are  limited 
compared  to  those  available  to  geographic  communities.  By  providing  opportunity  for  and  the  expectation 
of  social  interaction  among  participants,  we  purport  the  program  will  provide  the  plurality  needed. 

Autonomy  of  individual  members  within  the  community,  especially  within  an  academic  setting,  is 
important  to  foster.  We  will  encourage  thoughtful,  personal  postings  within  the  forum,  to  avoid  group-think 
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and  “me  too,  I agree”  contributions.  Students  will  receive  basic  instruction  on  netiquette  and  will  be 
encouraged  to  continually  address  evolving  group  norms  to  maintain  respectful  communication  and  to  build 
consensus.  We  use  individual  identity  in  place  of  autonomy  to  underscore  the  importance  ofboth  group  and 
individual  identities  within  a virtual  learning  environment. 

In  the  case  of  a virtual  community,  participation,  both  social  and  academic,  is  integral.  Without 
active  participation  in  discussions  and  other  class  activities,  the  learner  is  not  part  of  the  community; 
indeed,  the  learner  does  not  even  “exist.”  This  is  one  core  distinction  between  being  a passive  member  of  a 
physical  community  where  you  are  seen  and  yourpresence  is  noted  and  registered  in  the  minds  of  others.  In 
a virtual  community,  you  must  make  a concerted  effort  to  communicate  with  others  in  orderto  exist.  At  the 
same  time,  allowances  must  be  made  for  learners  to  shape  the  participation,  both  in  structure  (number/kind 
of  postings)  and  in  content  (managing  the  discussion  of  subjects  interesting  to  them). 

The  future  orientation  of  a learning  community  can  operate  at  a number  of  different  levels.  A 
stronger  community  bond  will  be  formed  when  a particular  cohort  goes  through  a number  of  courses 
together,  moving  toward  their  finishing  the  program  and  earning  a degree.  It  can  be  argued  that  a learning 
community  can  develop  within  the  constraints  of  a single  four-month  course,  but  it  is  much  more  likely  that 
students  will  form  long-lasting  academic  and  social  bonds  throughout  an  entire  program.  Visioning 
exercises  and  direction  of  learning  activities  (having  participants  describe  how  what  they  learned  will  help 
them  in  future  learning  and  in  their  work)  can  also  give  the  community  a focus  on  the  future.  In  our  case, 
the  community’s  view  of  the  future  may  be  limited  to  the  two  or  three  years  they  spend  in  the  program. 
However,  it  is  possible  that  they  will  continue  to  maintain  community  ties  once  they  have  earned  they 
degrees  and  are  working  again.  It  is  also  possible  that  members  of  the  Fall  2000  community  would  endup 
wanting  to  remain  part  of  the  Distance  Masters  community  afterthey  graduate,  and  would  like  to  integrate 
themselves  with  the  new  incoming  cohorts.  This  may  pose  particular  problems  of  negotiation  and  fit;  is 
there  a role  for  graduated  members  to  “return”  virtually  and  engage  with  students  working  through  the 
program? 

Schwier  notes  that  “the  nature  of  the  learning  can  be  broadly  defined  and  contextual”(p.  4)  but  is  a 
necessary  part  ofa  virtual  learning  community.  For  our  purposes,  the  learning  involved  is  more  specific 
and  structured;  the  cohort  moves  through  a set  of  core  courses  together,  in  a particular  order.  Our  goal  is  to 
foster  community  among  them  before  they  finish  the  first  year,  so  that  although  they  will  go  on  to  take 
other  courses  with  other  distance  learners,  they  will  not  only  maintain  ties  with  their  initial  cohort 
community,  but  will  also  have  learned  the  foundations  of  virtual  community  creation  and  will  use  these 
skills  in  other  classes.  We  have  changed  Schwier’s  term  learning  to  knowledge  generation. 

According  to  Schwier,  “communities  are  built  or  dismantled  by  those  in  the  communities,  not  by 
the  people  organizing  or  managing  them”  (p.  2).  As  they  mature,  communities  define  their  own  social  rules 
of  conduct  and  select  their  own  leaders,  assuming  ownership  of  their  governance  and  norms.  Learning 
communities,  note  Palloff  and  Pratt  ( 1 999),  exhibit  evidence  of  socially  constructed  meaning,  willingness 
to  critically  evaluate  the  work  of  others,  again  assuming  ownership  of  their  knowledge  creation  and 
sharing. 

Integration  of  all  of  these  elements  is  necessary  for  a strong  community.  Schwier  suggests 
creating  belief  statements  and  evolving  group  norms,  and  adhering  to  a learner-centered  philosophy  that 
“supports  individual  expression  while  building  a group  identity”  (p.  5). 

Finally,  technology  is  an  important  consideration  for  us:  although  it  is  thanks  to  certain 
technologies  that  virtual  community-building  is  even  possible,  there  are  certain  limitations  put  upon  the 
group  because  of  technology.  Although  it  is  the  conduit  for  discourse,  it  can  also  exclude  or  discourage 
people.  Tools  that  are  complicated,  unavailable  fora  certain  platform,  that  are  slow  and  cumbersome  can 
all  render  the  discussion  process  less  than  ideal,  and  members  who  do  not  actively  participate  essentially 
leave  the  community.  Although  Schwier  recommends  using  technology  compatible  with  older,  less  costly 
equipment  to  render  the  community  more  inclusive,  this  is  not  a concern  for  us. 

Based  on  Selznik’s  (1996)  seven  characteristics  and  Schwier’s  (in  press)  additional  three 
characteristics  of  community,  we  have  assembled  the  aforementioned  six  key  elements  of  community. 
From  these  elements,  we  define  community  as:  a group  of  people  who  are  brought  together  to  share  and 
generate  knowledge  in  a mutually  supportive  and  reciprocal  manner.  Its  characteristics  are  ownership, 
social  interaction,  group  identity,  individual  identity,  participation,  and  knowledge  generation. 
Furthermore,  integration  of  all  of  these  elements  is  necessary  for  a strong  community. 

Having  defined  some  of  the  particular  characteristics  of  a virtual  community,  we  will  now  turn  to  some 
basic  strategies  for  creating  community.  Palloff  and  Pratt  (1999)  recommend  these  steps: 

• Clearly  define  the  purpose  of  the  group 


• Create  a distinctive  gathering  place  for  the  group 

• Promote  effective  leadership  from  within. 

• Define  norms  and  a clear  code  of  conduct. 

• Allow  for  a range  of  member  roles. 

• Aiiow  for  and  facilitate  subgroups. 

• Allow  members  to  resolve  their  own  disputes  (p.  24) 

In  our  case,  many  of  these  steps  are  automatic,  but  they  should  still  be  given  careful  consideration.  For 
example,  the  general  purpose  of  the  community  is  defined  as  “the  Fall  2000  cohort  for  the  1ST  Distance 
Masters  program.”  However,  instructors  or  organizers  may  have  more  specific  goals  and  purposes  from  the 
beginning,  and  even  ifthey  do  not,  other  purposes  may  emerge  from  the  community  throughout  the  term. 
Pallof  and  Pratt  ( 1 999),  surprisingly,  do  not  put  much  emphasis  on  the  communicative  aspect  of  community 
without  which  a virtual  learning  community  cannot  exist. 

We  feel  that  one  of  the  most  important  indicators  of  a learning  community  is  the  first:  when  students 
communicate  not  only  on  an  academic  level  but  on  a personal  level.  Working  together  towards  the  goals  of 
the  course  is  what  they  are  “supposed”  to  be  doing.  When  they  begin  to  talk  about  their  personal  lives 
(families,  hobbies,  jobs),  their  triumphs  and  trials  with  being  a distance  student  (scheduling,  technical 
problems,  disagreement  with  pedagogy),  when  they  seek  each  other’s  counsel  for  other  areas  of  their  life 
(job  change,  which  elective  course  to  take  next,  family  issues),  this  is  the  point  at  which  we  feel  they  are 
comfortable  as  a community.  There  is  a good  chance  that  not  everyone  will  be  everyone  else’s  best  friend. 
However,  when  a majority  of  the  members  feel  they  are  in  a safe  enough  space  to  “speak  up”  about  things 
in  the  public  forum,  rather  than  in  individual  e-mail  messages,  then  this  is  evidence  of  a successful 
community.  There  may  be  a few  members  of  the  community  who  do  not  feel  that  the  Online  Cafe  is  an 
appropriate  place  to  discuss  non-academic  subjects,  and  it  is  the  role  of  the  mentor  and  the  community 
members  to  make  the  Cafe  a welcoming  place  for  this  type  of  discussion.  As  in  every  type  of  community, 
there  will  be  some  people  who  opt  out  of  certain  discussions,  or  even  out  of  all  “non-official”  discussion, 
but  this  is  quite  normal.  There  will  probably  be  smaller  communities  within  the  larger  online  class,  people 
who  form  bonds  and  discuss  the  course  work  and  their  lives,  but  not  on  the  general  forum.  These 
differences  can  appear  for  a variety  of  reasons;  Eastmond  (1995)  found  divisions  on  age,  gender, 
experience,  and  learning  style  lines.  However,  he  also  found  that  the  groups  often  transcended  age  and 
gender,  for  example,  two  characteristics  that  might,  in  a traditional  classroom,  be  impediments. 

The  final  step  in  creation  of  an  online  community  is  to  evaluate  whether  a community  has  formed,  and  if 
so,  in  what  ways  has  the  community  aspect  contributed  to  learning.  Our  project  will  address  methods  for 
performing  the  first  evaluation  of  whether  community  has  formed. 

Definitions 

We  will  examine  ways  to  use  certain  instructional  strategies  to  work  to  move  the  cohort  toward  a 
community.  We  suggest  encouraging  interaction  at  three  levels:  discussion,  cooperation,  and  collaboration. 

Cohort 

The  cohort  is  the  group  of  students  going  through  the  core  classes  as  a group.  They  may  have  an 
initial  connection,  such  as  a common  employer,  but  it  does  not  necessarily  constitute  a strong  bond. 

Discussion 

Discussion  is  the  basic  means  of  communication  in  an  online  format.  Students  must  participate  in 
discussion  to  have  any  sort  of  presence  in  the  class  whatsoever.  Discussion  can  be  focused  around  readings, 
lectures,  and  any  other  ideas  based  on  course  content  or  course  administration.  Discussion  can  occur 
asynchronously  in  the  SSF  or  via  e-mail,  or  synchronously  via  chat  rooms  or  telephone. 

Cooperation 

Cooperation  entails  students  working  in  groups  or  otherwise  dividing  up  tasks.  A machine 
metaphor  can  illustrate  cooperation  in  the  classroom:  different  parts  of  the  machine  perform  different 
functions  and  goals,  but  work  together  towards  a similar  end.  For  example,  students  may  divide  up  a 
project,  but  are  eventually  assigned  individual  grades  for  their  work.  Examples  of  cooperative  tasks 
include:  dividing  up  sections  of  a report  to  write  and  doing  peer  review  of  each  other’s  work. 
Collaboration 

Collaboration  is  the  most  integrated  form  of  group  work,  and  is  therefore  potentially  the  most 
difficult  and  the  most  rewarding.  In  the  case  of  collaboration,  the  group  members  work  toward  a common 
goal,  one  that  carries  a mutual  investment.  For  example,  students  may  each  work  on  every  part  of  the 
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report,  consulting  each  other  and  re-reading  each  other’s  edits.  They  are  invested  in  every  part  of  the 
project  because  they  will  share  a common  grade.  Examples  of  collaborative  tasks  include  group  writing  and 
creating  an  ID  model. 

Community 


A virtual  learning  community,  as  described  in  the  introduction,  is  one  of  the  ultimate  goals  of  the 
core  courses. 

The  three  levels  of  interactions  can  be  compared  by  several  characteristics,  as  in  the  table  below. 


Discussion 

Cooperation 

Collaboration 

Learning 

Information  transmission 

Knowledge  transmission 

Knowledge  generation 

Inquiry 

Individual  inquiry 

Delegation  of  tasks 

Common  inquiry 

Decision-making 

Agree  to  disagree 

Vote  (majority  rules) 

Social  negotiation  to 
consensus 

Goals/agendas 

Multiple  goals/  multiple 
agendas 

One  goal/  multiple 
agendas 

One  goal/  one  agenda 

Accountability 

Individual  accountability 

Individual  accountability 

Group  accountability 

Learning  relationship 

Complete  independence 

Partial  interdependence 

Complete 

interdependence 

Description  of  1ST  Core 

The  term  “Core”  is  used  in  the  1ST  department  to  denote  four  courses  that  all  graduate  students 
take  in  their  first  year  in  the  program.  Traditionally  R5 1 1 (2  credits),  R52 1 /522  (4  credits),  R580  ( 1 credit) 
are  offered  in  the  Fall  term;  R561  (3  credits)  is  offered  in  the  Spring.  It  is  usually  the  case  that  the  new 
students  (both  Masters  and  Doctoral)  take  these  classes  as  a group;  they  form  a cohort  that  goes  through  at 
least  the  first  year  of  courses  together.  The  cohort  identity  is  important  to  the  1ST  program,  and  it  is 
something  that  will  be  actively  cultivated  in  the  online  Masters  program.  Tangential  to  the  cohort  identity  is 
the  community-building  that  is  undertaken  to  integrate  new  students  into  the  1ST  program.  The  social 
aspect  of  the  community  is  nurtured  through  happy  hours,  the  1ST  picnic  in  the  fall  and  the  Follies  show  in 
the  spring,  and  informal  pairing  new  students  with  old  ones.  Academically,  the  1ST  community  is  built 
through  the  identity  of  the  Rookie  cohort,  through  the  rookies  taking  non-Core  classes  (R547,  Y520,  etc.) 
with  upper  year  students,  through  rookie  interaction  with  upper  year  AIs  in  Core  and  non-Core  classes,  etc. 
The  1ST  department  is  also  very  much  linked  to  its  alumni,  through  alumni  presentations  in  R580  (Grads 
with  Gigs)  and  networking  at  conferences. 

The  pedagogy  is  rooted  in  project-based  learning  and  team-based  work.  Much  of  the  learning  is 
hands-on,  and  students  often  work  with  real-world  clients.  There  is  a focus  on  an  integrated  curriculum  and 
many  of  the  courses  are  team-taught.  The  different  research  areas  of  the  faculty  (for  e.g.,  corporate  vs. 
higher  education  vs.  K- 1 2)  expose  all  students  to  multiple  academic  perspectives.  The  international  nature 
of  the  program  (approximately  one-third  of  the  students  are  non-U. S.  citizens)  exposes  all  students  to 
different  ways  of  learning  and  working.  Because  of  the  content,  there  is  an  emphasis  on  technological 
competence,  although  the  skill  levels  of  both  entering  and  graduating  students  vary  immensely.  Although 
the  use  of  technology  in  education  is  important  to  1ST,  technology  is  a means,  not  an  end,  and  its  use  is 
firmly  rooted  in  pedagogy. 

The  associations  that  1ST  has  with  other  departments,  including  Educational  Psychology, 
Language  Education,  the  Kelly  School  of  Business,  the  School  of  Library  Science,  etc.,  contribute  to  an 
integrated  and  interdisciplinary  academic  environment.  Most  of  these  departments  offer  online  courses  that 
can  be  used  by  Distance  Masters  students  as  electives. 


Core  Instructional  Strategies  and  Rationales 


Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Participatio 

Strategies 

Rationale 
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Students  participate  in  a face-to- 
face  orientation  on  campus. 

Face-to-face  interactions  allow  to  people  to  i 
create  strong  initial  bonds,  which  will  lead  to 
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a greater  sense  of  community  right  from  the 
beginning. 

/ 

/ 

/ 

Students  will  leam  about  online 
communication,  including  rules 
of  netiquette 

Online  communication  is  vastly  different  from 
more  traditional  forms  of  communications 
(Black,  1995). 

✓ 

y 

/ 

Students  will  undergo  training  in 
using  SiteScape  Forum,  e-mail, 
majordomo  creation,  basic  web 
searches,  and  MS  Word  for 
collaborative  writing  purposes. 

To  help  reduce  barriers  to  effective  learning 
and  establishing  social  relationships, 
participants  should  be  given  the  opportunity  to 
build  confidence  and  competence  with  the 
distance  education  process  and  supporting 
technologies  (IDE,  2.2). 

v' 

y 

Students  will  post  photos  of  in 
SiteScape  Forum  at  Orientation. 

Connecting  people’s  names  and  faces  is  a first 
big  step  to  forming  bonds. 

✓ 

/ 

y 

/ 

Students  will  participate  in  a 
content-based  group  project  that 
requires  that  they  negotiate  the 
exact  content. 

People  will  form  strong  personal  and 
academic  bonds  through  shared  adversity 
(Ruhleder,  1999). 

/ 

y 

Students  will  be  required  to  eat 
lunch  as  a group  two  days  during 
Orientation. 

People  who  have  a social  connection  to  the 
group  will  work  better  together  (Palloff  & 
Pratt,  1999). 

/ 

y 

Students  will  be  given  the 
opportunity  to  participate  in  at 
least  two  evening  social  activities. 

People  who  have  a social  connection  to  the 
group  will  work  better  together  (Palloff  & 
Pratt,  1999) 

y 

✓ 

First  posting  should  be  a non- 
graded/non-credited  assignment 
(e.g.  biography). 

Students  need  non-threatening,  interesting 
ways  to  begin  creating  online  community 
(Funaro,  1999). 

✓ 

y 

/ 

y 

✓ 

Create  an  online  cafe  that  will 
serve  as  a non-course-specific 
conversation  area  to  encourage 
off-task  communication. 

People  need  distinctions  between  work  and 
play  (Palloff  & Pratt,  1999). 

y 

V 

y 

Encourage  instructor  and  distance 
mentors  to  participate  in  social 
interactions,  especially  in  the 
early  stages  of  the  course. 

Social  interactions  between  and  among 
learners  enrich  the  learning  community  and 
should  be  supported  in  the  instructional  design 
of  the  course  (IDE,  2.5). 

y 

y 

y 

y 

Students  will  be  encouraged  to 
share,  in  the  online  cafe, 
information  about  their  non- 
academic  lives,  for  example, 
offering  mutual  support  in  term  of 
how  they  are  keeping  up  with 
their  job  and  school  schedule. 
Students  should  be  encouraged  to 
offer  successful  strategies  to  the 
class. 

Reciprocity  and  help  are  two  important 
hallmarks  of  community.  Students  who  take 
an  interest  in  each  other’s  well  being,  both 
academic  and  social,  will  have  more  of  a 
support  system  of  peers  than  those  who  do  not 
(Wellman  & Gulia,  1999). 

R511  Section 

Description  of  R511  (from  course  syllabus) 

R5 1 1 , Instructional  Technology  Foundations  I,  is  a two-credit  course  that  has  historically  been 
offered  each  fall  semester.  This  course  is  required  by  all  1ST  Masters  students  and  is  typically  taken 
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concurrently  with  R52 1 /522,  Instructional  Design  and  Development,  and  R580, 1ST  Colloquium.  It  is  team- 
taught  by  two  faculty  members  and  one  graduate  assistant  who  has  taken  the  course. 

The  overall  objective  of  this  course  is  to  provide  a comprehensive  introduction  to  the  field  and 
profession  of  Instructional  Technology  (IT).  Since  most  entering  1ST  students  come  from  fields  other  than 
instructional  technology,  P.5!  1 gives  newcomers  a sense  of  history  and  an  explanation  of  how  the 
components  of  the  field  fit  together.  There  is  a particular  emphasis  on  the  evolution  of  the  “big  ideas”  of 
the  field. 

In  the  onsite  version  ofR51  1,  class  meetings  occur  once  per  week  in  2-hour  sessions.  Directed 
readings  compiled  in  a course  packet  are  provided  as  practical  resources  to  support  assignments  and  class 
discussion  activities  in  the  course.  Most  class  periods  are  divided  into  two  portions: 

1 ) During  the  first  hour,  each  of  the  three  instructors  facilitates  a group  discussion  among  1 5-20  students 
about  assigned  readings.  2)  The  remaining  portion  of  the  class  time  is  devoted  to  further  lecture  and 
clarification  about  topics  contained  in  the  readings. 

Students  are  graded  according  to  participation  in  class  discussion,  personal  synthesis  and 
reflection  (as  noted  in  weekly  minute-papers  collected  at  the  end  of  each  class),  three  individual  written 
essays  (one  team-based,  two  individual),  and  a final  exam  or  written  essay. 

R511  Instructional  Strategies  and  Rationales 


Ownership 

Know.  Gen. 

Ind.  Identity 

Shared  Iden. 

Social  Inter. 

Particinatio 

Strategies 

Rationale 

A fundamental  element  for  success  for 
the  distance  students  is  an 
understanding  of  the  key  expectations 
how  much  time  the  course  will 
require 

the  level  of  performance  that  is 
expected  of  them 
the  demands  that  participating  in 
the  core  will  have  on  their  time. 

Students,  but  especially  students  learning 
at  a distance,  need  to  have  expectations, 
assumptions,  deadlines,  etc.,  made 
explicit  and  kept  clear  (Palloff  & Pratt, 
1999). 

Understanding  and  respecting  expecta- 
tions for  participation  and  performance 
will  be  critical  to  the  students’  success. 
Taking  Core  online  will  be  more 
demanding  than  doing  it  face-to-face. 

A 

y 

Instructors  will  assign  discussion  roles 
(facilitator,  summarizer,  devil’s 
advocate,  etc.)  to  encourage  shy 
members  and  force  students  to  think  in 
different  ways  about  the  material  and 
about  the  discussion  of  the  material. 

Students  should  be  challenged  to  engage 
the  material  from  different  perspectives; 
different  roles  improve  learner-learner 
interaction  and  improve  learner-material 
interaction. 

/ 

/ 

Students  will  be  expected  to  take  part 
in  regular  peer  reviews  by  critically 
evaluating  each  other’s  papers. 

It  is  important  to  develop  a critical  eye 
towards  other  community  members’ 
work. 

Each  week,  someone  from  each  group 
will  summarize  their  group’s 
discussion  and  post  the  results  for  the 
other  groups  to  read. 

Bringing  from  small  groups  to  the  larger 
group  provides  for  more  viewpoints  and 
better  discussion. 

/ 

✓ 

Students  will  be  divided  into  3-4  small 
groups  for  discussion  of  readings  and 
course  projects. 

Small  groups  facilitate  better  discussion 
(Hiltz,  1998)  for  learner-material 
interaction. 

✓ 

✓ 

Students  will  fill  out  weekly  “1- 
minute  evaluation”  web  form,  to 
instructor  only.  Possible  topics  include 
what  you  liked/disliked  about  the 
week’s  work,  how  you  can  transfer 

To  better  assimilate  and  process  what 
they  have  learned,  students  require  a 
forum  to  critically  reflect  on  the  material 
and  on  themselves  as  learners  (Palloff  & 
Pratt,  1999).  Keeping  in  touch  with  the 
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this  knowledge  to  your  work,  and 
generally  how  you  are  feeling. 

professor  improves  learner-faculty 
interaction. 

y 

V 

✓ 

Instructors  will  require  high-quality 
online  interactions  with  peers  and 
discussions  of  readings  by  making  a 
portion  of  the  grade  dependent  on  it. 
(We  recommend  at  least  25%). 

Effective  learning  environments  should 
provide  frequent  and  meaningful 
interactions  among  learners.  (IDE,  2.1) 
Good  practice  encourages  cooperation 
among  students  (Chickering  & Gamson, 
1987). 

/ 

/ 

Instructor  and/or  mentor  will  model 
ways  to  produce  lively,  constructive 
discussion:  questions  should  be  open- 
ended,  but  focused  on  students’ 
interpretation  of  the  text. 

One  of  the  best  ways  to  keep  discussion 
on  topic  and  students  motivated  is  to 
participate  actively  in  the  conversation 
(Beaudin  1999). 

/ 

✓ 

✓ 

Instructor  will  point  out  excellent 
discussion,  postings,  interactions,  etc. 
of  other  students  to  continually 
promote  high  expectations  and  model 
good  interaction. 

Good  practice  encourages  prompt 
feedback  (Chickering  & Gamson,  1 987). 
Faculty-learner  interaction  improved  by 
attentive  professor. 

v 

As  needed,  instructor  will  revisit 
netiquette  and  general  interaction 
issues,  and  stresses  the  importance  of 
interacting  in  a respectful  way.  Have 
the  community  develop  group  norms 
based  on  emergent  issues. 

Social  negotiation  leads  to  the  creation  of 
a safe  space,  which  is  essential  for 
learning  (Palloff  & Pratt,  1999). 

✓ 

Students  will  be  expected  to  check 
SiteScape  Forum  and  e-mail  every 
two  days  and  post  quality 
contributions  at  least  twice  a week. 
Participation  points  will  be  calculated 
based  on  these  postings. 

Because  of  the  nature  of  the  evolving 
discussion,  students  should  be  constantly 
engaged  in  the  course,  without  any 
lengthy  absences  from  discussion. 
(Caldwell  & Taha,  1993) 

✓ 

S 

V 

V 

The  instructor/AI  should  make  contact 
with  students  who  are  not  actively 
participating  to  find  out  why  and 
address  their  concerns. 

Students  need  to  actively  feel  like  they’re 
part  of  the  community,  and  that  the 
instructor  is  interested  in  their  well-being, 
academic  or  otherwise  (Palloff  & Pratt, 
1999). 

Students  will  work  together  at  all  three 
levels  of  interaction: 

Discussion 

Cooperation 

Collaboration 

In  order  for  a newly-formed  cohort  to 
move  to  community,  they  must  change 
the  quality  of  their  interactions.  The 
community  should  move  toward 
successful  use  of  collaboration,  in 
addition  to  the  continued  use  of  group 
discussions  and  cooperative  tasks. 

R511  Assignment  Specifications 

Based  on  the  existing  assignments  for  R5 11,  we  have  developed  a set  of  projects  and  assignments 
that  will  both  address  the  traditional  content  of  the  course,  and  build  community  based  on  the  discussion, 
cooperation,  and  collaboration  model.  Where  we  realize  that  collaboration  is  a more  rich  form  of  thinking 
and  working  together,  we  also  emphasize  the  necessity  for  students  to  work  at  all  three  levels  of  interaction 
throughout  the  course. 
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Discussion 

At  the  beginning  of  the  semester,  students  will  be  divided  into  readings  discussion  groups  of  3-4 
people.  For  purposes  of  community  and  continuity,  they  will  remain  in  these  groups  throughout  the 
semester. 

In  SiteScape  Forum,  a team  will  be  created  for  each  readings  discussion  group.  The  group  will  manage 
that  space,  and  can  create  folders  for  each  week’s  readings  if  they  so  choose. 

The  students  will  be  expected  to  discuss  the  week’s  readings  in  their  respective  folders.  Each  student 
should  post  at  least  twice  each  week. 

The  role  of  facilitator  in  each  discussion  group  will  rotate  from  week  to  week.  The  facilitator  must  start 
the  conversation,  and  engage  group  members  to  participate. 

The  role  of  summarizer  in  each  discussion  group  will  rotate  from  week  to  week.  At  the  end  of  the 
week,  the  summarizer  must  condense  the  group’s  main  discussion  points,  and  post  them  to  the  class 
forum  (outside  the  team  space). 

There  will  be  a separate  folder  in  SiteScape  Forum  for  discussion  of  the  week’s  lecture  or  class 
activity.  These  posting  requirements  will  be  determined  at  a latertime,  dependent  on  the  format  ofthe 
course  lecture  material. 

Cooperation 

“Letter  Home”  Paper 

Students  will  review  each  other’s  papers  in  formative  stages.  For  the  “letter  home”  assignment, 
students  will  post  outlines  and  rough  drafts  by  set  deadlines,  and  a selected  group  of  peers  (ideally  from 
outside  their  reading  group)  will  have  to  read  and  give  feedback  on  them.  The  rationale  for  a number  of 
small  deliverables  leading  to  the  final  paper  is  that  distance  students  traditionally  need  regular  deadlines 
and  prompt  feedback. 

Final  Exam  Study  Guide 

Students  can  still  help  each  other  out  on  breaking  down  the  study  guide  and  elaborating  on  certain 
sections  of  it.  This  could  be  left  open  for  students  to  determine,  just  as  in  the  traditional  R51  1 class.  Simply 
make  the  suggestion  to  the  students  that  they  may  want  to  work  together  on  fleshing  out  the  study  guide, 
and  leave  it  to  them  to  decide  how  they  want  to  do  it. 

Collaboration 

ID  Model  Paper 

The  students  will  collaborate  on  the  ID  model  paper  as  in  the  traditional  R51 1 class  (using  their 
reading  discussion  teams  as  the  groups).  In  the  distance  version,  however,  it  will  be  critical  that  this  process 
be  divided  into  small  deliverables.  For  instance,  the  students  might  be  required  to  break  down  the  task  into 
the  following  deadlines: 

Week  One:  Each  group  member  must  post  initial  ideas  of  possible  models  to  evaluate  or  create.  This  is 
not  in  any  formal  structure  - just  a brain  dump.  Each  group  member  must  read  and  respond  to  the 
discussion. 

Week  Two:  Group  must  decide  on  a model  and  begin  explicating  the  model  and  describing  its 
strengths/weaknesses.  All  group  members  should  be  posting  during  this  week. 

Week  Three:  Someone  in  the  group  should  summarize  the  discussion  into  a paper  outline.  Another 
group  member  should  develop  a paper  draft.  One  or  two  group  members  should  make  suggested 
changes  and  revisions.  The  final  group  member  should  write  the  final  draft  and  post  it. 

“ism”  Debate 

Students  will  participate  in  group  debates  revolving  around  behaviorism,  cognitivism,  and 
constructivism.  Ideally,  students  will  be  placed  into  3 groups  that  are  different  from  their  reading  groups. 
Each  group  will  be  assigned  one  ofthe  “isms”  to  represent  in  the  debate.  Here,  a proposed  timeline  forthe 
debate: 

Week  One:  Individuals  will  write  an  informal  short  paper  (one  page  - perhaps  even  as  a bulleted  list) 
highlighting  the  major  strengths  oftheir“ism”  as  it  applies  to  distance  education  courses  and  will  post 
it  for  their  teammates.  Next,  the  team  will  enumerate  possible  rebuttals  from  the  other  groups  and 
responses  to  those  arguments.  The  first  week’s  discussion  and  postings  will  all  take  place  inside  a new 
folder  established  for  that  team. 

Week  Two:  One  student  from  each  group  will  post  an  argument  about  why  theirposition  is  the  best  to  a 
debate  folder  open  to  the  whole  class.  Each  group  will  respond  to  each  of  the  other  groups. 
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Week  Three:  Debate  will  continue. 

Week  Four:  Each  individual  will  write  a brief  reflection  on  how  their  opinion  changed  throughout  the 
debate. 

Checklist  for  R511  Instructor/Mentor 
Orientation 

□ Attend  Sunday  night  dinner  with  new  DE  students. 

Q Participate  in  2-hour  R51 1 class  welcome  session. 

Beginning  of  Semester 

□ Create  teams  in  SiteScape  Forum  for  each  readings  discussion  group.  Using  a naming  structure 
like  jewels  (Opal  team,  Ruby  team,  etc.)  is  an  easy  identifying  factor. 

o Divide  students  into  the  groups  evenly.  Make  sure  the  instructor  and  GA  are  listed  as 
members  of  all  teams. 

□ In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R51 1 Lecture  and  Class 
Activity  Discussion.” 

Q In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R5 1 1 Resources  & 
Tidbits” 

Q Create  a class  majordomo. 

Weekly 

O Check  that  all  class  members  have  posted  at  least  twice  about  the  readings. 

o If  not,  make  decision  about  contacting  that  person  via  e-mail. 

O Check  that  all  class  members  have  posted  about  the  lecture/class  activity, 
o If  not,  make  decision  about  contacting  that  person  via  e-mail. 

□ Post  some  comments  to  the  Online  Cafe.  This  could  be 
o News  stories 

o Responses  to  other  students  v 

o Encouragement 
o Personal  comments 
o IST/DE  news 

O Reply  to  at  least  2 postings  a week,  to  encourage  students  to  post  thoughtful  responses  and  to 
show  that  you  are  present  and  actively  following  the  discussions. 

Before  the  “ism”  Project 

□ In  SiteScape  Forum,  create  the  following  three  teams:  Behaviorism,  Cognitivism,  and 
Constructivism. 

o Divide  students  equally  among  the  three  teams.  Make  sure  the  instructor  and  GA  are  listed  as 
members  of  all  teams. 

Before  the  “Letter  Home”  Paper 

□ In  SiteScape  Forum,  create  the  following  five  teams:  Peer  Review  Group  1,  Peer  Review  Group 
2,  Peer  Review  Group  3,  Peer  Review  Group  4,  and  Peer  Review  Group  5. 

o Divide  students  equally  among  the  five  teams.  Make  sure  the  instructor  and  GA  are  listed  as 
members  of  all  teams. 

R52 1/522  Section 

Description  of  R521/522  (from  course  syllabus) 

R52 1/522,  Instructional  Design  and  Development,  is  a four-credit  course  that  has  historically  been 
offered  each  fall  semester  in  an  onsite  format.  This  course  is  required  by  all  1ST  MS  students  and  is 
typically  taken  concurrently  with  R511,  Instructional  Technology  Foundations  I,  and  R580,  1ST 
Colloquium.  It  is  team-taught  by  at  least  two  faculty  members  and  one  or  two  graduate  assistants  who  have 
taken  the  course  themselves. 

Major  content  and  experience  objectives  of  R52 1/522: 

Knowledge  of  instructional  design  principles 

Knowledge  and  application  of  the  ADDIE  model  of  instructional  design  and  development 


59 


- *‘~i 

> 4 1 


*—) 

* > 


Understanding  and  application  of  simple  formative  evaluation  processes 

Ability  to  recognize  and  employ  fundamental  principles  and  experiences  in  team-based  approach  to 
project  work 

Pedagogical  methods  used  in  R52 1/522: 

Task-oriented  learning  through  “authentic”  projects 
Diverse,  team-based  project  groups 
Mentor/coach-based  instruction  for  project  team  support 
Structured  timeline  of  deadlines  and  deliverables 

Independent  learning,  i.e.,  students  take  responsibility  for  their  own  learning 

Assignments  with  specific  criteria  that  engage  students  in  learning  specific  course  content,  with  leeway 
given  for  students  to  identify  their  own  topics 

Most  of  the  learning  in  the  course  occurs  within  the  context  of  projects  and  situations  similar  to 
those  that  instructional  designers  encounter  in  professional  work.  Projects  are  sequenced  such  that  the 
processes  and  principles  learned  in  the  first  ones  provide  foundation  of  understanding  and  competence  for 
progressively  more  complex  ones  that  follow.  This  progression  of  increasingly  elaborated  projects 
continues  through  the  academic  year  into  R56 1 , Evaluation  and  Change  Management,  and  is  intended  to 
carry  on  throughout  the  student’s  academic  experiences  in  completing  the  1ST  MS  program. 

In  the  onsite  version  of  R52 1/522,  class  meetings  occurtwice  per  week  in  2.5  -hour  sessions.  Class  sessions 
involve  one  or  more  of  a variety  of  activities,  including  lectures  or  presentations  about  specific  topics, 
readings  discussions,  project  group  meeting  time,  group  project  presentation,  or  hands-on  design  activities. 
Directed  readings  compiled  in  a course  packet  are  referenced  as  practical  resources  to  support  projects  and 
class  discussion  activities  in  the  course. 

The  instructors  believe  that  people  learn  best  when  they  are  highly  motivated  and  actively  engaged 
in  learning  tasks,  that  learning  is  most  useful  when  it  is  directly  related  to  learner  needs.  Thus,  students  are 
expected  to  take  responsibility  for  their  own  learning.  The  course  begins  with  a fair  amount  of  guidance 
from  the  instructors,  in  terms  of  what  information  to  access  and  how  to  facilitate  personal  learning,  then 
gradually  decreases  that  guidance  to  require  students  to  actively  seek  resources  on  their  own  to  perform  the 
assigned  tasks. 

Major  projects  in  R52 1/522  are  completed  by  groups  of  three  students,  each  mentored  by  an 
assigned  instructor  “coach.”  To  perform  most  satisfactorily  in  the  course,  students  must  spend  many  hours 
per  week  outside  of  class  developing  and  completing  these  projects.  At  the  completion  of  a project,  each 
member  of  a given  group  is  awarded  the  same  grade  (a  “group  grade”)  as  his/her  teammates. 
Approximately  twenty  percent  of  that  grade  is  awarded  for  the  deliverable  produced  in  the  project  (e.g.,  the 
instructional  tool  developed  and  a design  report),  whereas  the  remaining  portion  of  the  grade  is  awarded 
according  to  the  way  members  worked  within  the  team  setting.  Some  students  come  into  the  program  with 
extensive  background  in  true  teamwork,  but  most  do  not.  Thus,  the  instructors  devote  a portion  of 
instructional  time  early  in  the  semester  toward  preparing  students  for  the  team  experience.  Throughout  the 
duration  of  each  project,  group  coaches  continue  to  offer  advice  and  guidance  for  the  team  process. 


R52 1/522  Instructional  Strategies  and  Rationales 
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Learners  will  be  divided  into  3-4 
groups  for  discussion  of  readings  and 
course  projects. 

Small  groups  facilitate  better  discussion. 
(Hiltz,  1998) 

V 

V 

Provide  criteria  that  define  appropriate 
course  topics,  leaving  room  for  choice 
and  opportunities  to  leverage  work- 
related  projects  as  course  projects. 

• Learners  select  a topic  and 
procedure  for  project. 

• Each  team  selects  4 readings  to 

In  order  to  build  community,  learners 
need  ownership.  (Schwier,  in  press) 
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read  and  summarize  for  the  whole 
class  (for  discussion) 

y 

y 

y 

Each  week,  someone  from  each  group 
will  summarize  their  group’s 
discussion  and  post  the  results  for  the 
other  groups  to  read. 

Bringing  from  small  groups  to  the  larger 
group  provides  for  more  viewpoints  and 
better  discussion. 

y 

y 

y 

Students  will  be  divided  into  3-4  small 
groups  for  discussion  of  readings  and 
course  projects. 

Small  groups  facilitate  better  discussion 
(Hiltz,  1998)  for  learner-material 
interaction. 

y 

y 

y 

Instructional  activities  will  require  the 
learner  to  actively  participate  in  the 
acquisition  and  processing  of 
educational  content. 

• Team-based  authentic  projects 
where  the  learners  learn  by  doing. 

• Discussing  readings  online  and 
role-playing. 

• Require  high-quality  online 
interactions  with  peers  and 
discussions  of  readings  by 
making  a portions  of  the  grade 
dependent  on  it  (25% 
recommended) 

• Groups  will  be  responsible  for 
posting  some  of  their  work  in 
progress  (e.g.,  each  group  posts  a 
description  of  a different  aspect  of 
ADDIE). 

• Instructor  should  use  open-ended 
questions  to  encourage  dialogue. 

• Ask  learners  to  provide  URLs  that 
enhance  learning. 

To  better  assimilate  and  process  what 
they  have  learned,  students  require  a 
forum  to  critically  reflect  on  the  material 
and  on  themselves  as  learners  (Palloff  & 
Pratt,  1999).  Keeping  in  touch  with  the 
professor  improves  learner-faculty 
interaction. 

y 

y 

Instructor  will  phone  each  learner 
before  class  begins.  (Spear  & Bruce, 
1997) 

One-on-one  verbal  communication 
between  learner  and  instructor  solidifies 
relationship. 

y 

y 

Establish  a virtual  office  hour:  one 
hour  where  instructor  will  be  available 
for  online  chats,  office  phone  calls,  or 
e-mail.  Inform  learners  of  the  faculty 
member’s  expected  e-mail  or 
voicemail  response  time,  e.g.,  within 
24  hours,  twice  a week,  etc.  (Spear  & 
Bruce,  1 997)  (Spear  & Bruce,  1997) 

Students  like  to  know  the  professor  is 
available  at  a particular  time  to  address  e- 
mail  concerns. 

y 

y 

Instructor  will  be  proactive,  following 
up  on  the  learner  who  is  not 
participating  in  chats,  discussions,  etc. 

In  the  distance  format,  it  is  easy  for 
students  to  lose  touch  with  the  class  and 
slowly  drop  out.  Active  intervention 
from  the  instructor  can  lessen  attriction. 

y 

y 

y 

y 

Conduct  a phone  conference  with  each 
team  at  least  once  during  the 

Verbal  communication  between  the  team 
and  instructor  solidifies  relationship  and 
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development  of  each  project. 

makes  for  easier  clarification. 

Invite  other  professors  to  lead 

Students  will  appreciate  input  from 

discussions  (Davies  and  Reigeluth  on 

experts  in  the  field.  The  will  feel  less 

concept  learning)  where  learners  can 

isolated  from  the  rest  of  the  department 

interact  directly  with  experts  to 

when  they  can  interact  with  other 

deepen  understanding. 

instructors. 

R521  Assignment  Specifications 

Based  on  the  existing  assignments  for  R52 1 , we  have  developed  a set  of  projects  and  assignments 
that  will  both  address  the  traditional  content  of  the  course  and  build  community  based  on  the  discussion, 
cooperation,  and  collaboration  model.  The  students  will  collaborate  on  production  projects,  discuss 
readings  or  lecture  topics,  and  reflect  on  activities  and  experiences  throughout  the  course.  While  we  realize 
that  collaboration  is  the  richest  form  of  thinking  and  working  together,  we  also  emphasize  the  necessity  for 
students  to  work  at  all  three  levels  of  interaction  throughout  the  course. 

Discussion 

Readings 

Discussion  activities  centered  around  the  course’s  major  themes  (e.g.,  usability,  design, 
evaluation). 

At  the  beginning  of  the  semester,  students  will  be  divided  into  readings  discussion  groups  of  5 people 
(different  from  those  in  their  project  groups).  For  purposes  of  community  and  continuity,  they  will 
remain  in  these  groups  throughout  the  semester. 

In  SiteScape  Forum,  a team  will  be  created  for  each  readings  discussion  group.  The  group  will  manage 
that  space,  and  can  create  folders  for  each  week’s  readings  if  they  so  choose. 

The  students  will  be  expected  to  discuss  the  assigned  readings  in  their  respective  folders.  Each  student 
should  post  at  least  twice  each  week. 

The  role  of  facilitator  in  each  discussion  group  will  rotate  from  assignment  to  assignment.  The 
facilitator  must  start  the  conversation,  and  engage  group  members  to  participate. 

The  role  of  summarizer  in  each  discussion  group  will  rotate  from  assignment  to  assignment.  At  the 
end,  the  summarizer  must  condense  the  group’s  main  discussion  points,  and  post  them  to  the  class 
forum  (outside  the  team  space). 

Cooperation 

Group  projects 

• At  the  beginning  of  each  project,  students  will  be  divided  into  groups  of  three.  Each  group  will  work 
collectively  to  complete  its  own  project.  A team  “coach”  (an  instructor  or  graduate  assistant)  will  be 
assigned  to  each  group  to  offer  advice  and  guidance  for  the  team  process. 

• In  SiteScape  Forum,  a team  will  be  created  for  each  project  group  (including  the  course  instructors  and 
mentors).  The  group  will  manage  that  space. 

• Groups  will  be  required  to  post  all  team  meeting  summaries  and  other  artifacts  of 
their  team  processes  on  the  forum. 

Collaboration 

Group  projects 

For  each  project,  the  team  will  be  intentionally  diverse  in  gender,  nationality  and/or  job  background  as 
much  as  possible  to  encourage  multiple  points  of  view. 

Projects  will  be  assigned  group  grades,  a large  portion  of  which  is  assigned  to  the  “group  process.” 

Give  project  rubrics,  teams  will  be  encouraged  to  brainstorm  possible  topics  and  come  to  consensus  to 
identify  their  own  topics  for  projects. 
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Teams  will  engage  in  formative  peer  reviews  of  each  others’  projects  and  materials  for  projects 
throughout  the  course. 

Lectures  and  course  topics  will  be  presented  by  different  instructors  throughout  the  course,  providing  a 
model  of  collaboration  for  students. 

Checklist  for  R521  Instructor/Mentor 
Orientation 

□ Attend  Sunday  night  dinner  with  new  DE  students. 

□ Coordinate  a team-based  project  that  emulates  the  required  peer  interaction  and  interdependence 
and  time-limited  working  tensions  of  R521  production  projects. 

□ Post  expectations  (time,  participation,  assignments,  dates) 

Beginning  of  Semester 

□ Create  teams  in  SiteScape  Forum  for  each  readings  discussion  group. 

o Divide  students  into  the  groups  evenly.  Make  sure  the  instructor  and  GA  are  included  as 
members  of  all  teams. 

□ In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R521  Lecture  and  Class 
Activity  Discussion.” 

□ In  SiteScape  Forum,  create  a Discussion  & Document  Forum  entitled  “R521  Resources  & Tidbits” 

□ Create  a class  majordomo  listserv  and  direct  all  class  members  to  subscribe  to  it. 

Weekly 

G Post  some  comments  to  the  Online  Cafe.  This  could  be 
o News  stories 
o Responses  to  other  students 
o Encouragement 
o Personal  comments 
o This  week  in  1ST 
o Post  reflection  questions  each  week 
Beginning  of  Each  Project 

□ Create  teams  in  SiteScape  Forum  for  each  project  group. 

o Divide  students  into  the  groups  evenly.  Make  sure  the  instructor  and  GA  are  included  as 
members  of  all  teams. 

G Direct  each  team  to  construct  and  post  its  own  individualized  strategies  and  timeline  for 
conducting  the  team  process  and  completing  its  project. 

Throughout  Project 

□ Check  that  each  project  team  is  posting  evidence  of  cooperative  work  on  project  at  least  once  per 
week. 

o If  not,  make  decision  about  contacting  that  group  via  e-mail. 

□ Reply  to  at  least  1 or  2 postings  a week  per  group,  to  encourage  students  to  post  thoughtful 
responses  and  to  show  that  you  are  present  and  actively  following  the  discussions 

□ Check  that  all  team  members  are  participating  at  least  once  every  two  weeks  within  their  own 
project  teams 

End  of  Project 

G Review  reflection  essays  from  each  team  member  about  lessons  learned  from  the  production  and 
team  processes 

G Collect  peer  grading  of  team  members’  participation  within  each  team 

Evaluation 

The  final  step  in  the  creation  of  a learning  community  in  these  courses  is  to  evaluate  whether  such 
a community  has  formed  and,  if  so,  in  what  ways  the  community  aspect  has  contributed  to  learning.  We  are 
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basing  our  strategies  for  evaluating  the  success  of  community-building  in  these  courses  on  Palloff  & Pratt’s 
( 1 999)  indicators  that  an  online  community  has  been  forming: 

Active  interaction  involving  both  course  content  and  personal  communication. 

Collaborative  learning  evidenced  by  comments  directed  primarily  student  to  student  rather  than  stuueni 
to  instructor. 

Socially  constructed  meaning  evidenced  by  agreement  or  questioning,  with  the  intent  to  achieve 
agreement  among  students. 

Sharing  of  resources  among  students 

Expressions  of  support  and  encouragement  exchanged  between  students,  as  well  as  willingness  to 
critically  evaluate  the  work  of  others,  (p.  32) 

The  course  evaluations  will  take  two  forms:  Formative  evaluations  are  undertaken  throughout  the 
course  so  that  necessary  adjustments  in  course  delivery  and  activities  can  be  identified  and  made. 
Summative  evaluations  are  performed  at  the  conclusion  of  the  course  to  measure  final  learning  outcomes 
and  student  satisfaction.  Both  forms  provide  fundamental  indicators  of  the  overall  success  of  the  course  and 
its  participants  in  meeting  the  initially  stated  objectives.  Palloff  & Pratt  recommend  employing  evaluations 
over  three  distinct  elements  of  a online  course:  student  performance  and  learning,  effectiveness  of  the 
course  in  supporting  student  learning  objectives,  and  overall  student  experiences  of  the  students  in  working 
in  an  online  environment.  For  our  purposes  of  assessing  community  formation,  the  emphasis  on  student 
performance  is  most  the  most  important  factor  on  which  to  focus. 

We  have  stated  already  that  two  key  indicators  that  the  evolution  of  a community  has  occurred  are 
evidence  of  participants  accepting  ownership  of  the  community  and  realizing  a shared  identity.  The 
metaphor  of  scaffolding  activities  and  course  strategies  as  mechanisms  to  foster  community  implies  that  the 
instructor  provides  models  and  activities  to  course  members  through  which  they  exercise  community-like 
tasks  and  interactions.  These  scaffolds  are  erected  as  temporary  measures  to  support  the  desired  behavioral 
outcomes  until  observed  behavior  indicates  they  are  no  longer  used  or  needed,  then  they  are  gradually 
removed.  Concurrently,  formative  evaluation  that  measures  indicators  of  the  extent  to  which  online 
community  is  occurring  becomes  the  key  factor  in  determining  the  necessity  and  lifespan  of  each 
scaffolding  device. 

Suggested  methods  for  formatively  assessing  the  level  of  online  community  throughout  the 
duration  of  the  courses  are  as  follows: 

Continually  monitor  the  amount,  type  and  effectiveness  of  discussion  in  all  media,  particularly  student- 
to-student  discussion 

Administer  periodic  interviews  and  web-based  questionnairesto  students  to  gather  qualitative  feedback 
about  reactions  to  the  level  of  community  they  are  experiencing  and  its  usefulness  to  their  learning 

Look  for  evidence  within  all  communication  media  of  resource  sharing  and/or  inter-community 
encouragement  or  support 

Compare  progression  of  reflective  essays  of  students  to  identify  evolution  of  self-assessments  that 
indicate  personal  commitment  to  the  community  or  deepening  of  learning  and  thought  about  key  issues 
discussed  among  members 

We  do  not  anticipate  that  a mature  community  will  have  been  generated  from  this  one  semester 
alone.  However,  we  do  expect  that  the  R52 1 and  R5 1 1 experiences  of  these  students  will  create  a solid 
foundation  of  an  infant  community  that  will  continue  evolving  throughout  their  career  in  the  1ST  DE  MS 
program.  Summative  evaluation  in  the  context  of  assessing  community  building  is  useful  for  determining 
the  overall  effectiveness  of  the  online  community  environment  on  the  students’  experiences  both  during 
these  courses  and  in  future  ones. 

Suggested  methods  of  summative  evaluation  are  as  follows: 

Compare  pre-and  post-course  attitudes  of  students  regarding  confidence  with  working  collaboratively 
with  a distributed  or  online  project  team 

Compare  pre-  and  post-course  opinions  of  students  regarding  their  comfort  levels  with  and  reactions  to 
collaborative  projects 

Assign  a final  reflective  essay  in  which  students  describe  a personal  action  plan  for  applying  the 
experiences  and  knowledge  gained  through  the  course,  specifically  those  relating  to  collaboration  and 
communities 
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Perform  longer-range  (e.g.,  2-3  months  later)  follow  up  interviews  and  surveys  with  students  that 
engage  them  in  reflection  on  the  impact  of  community  and  collaboration  on  courses  taken  after  R521 
and  r5  1 1 

Finally,  we  intend  these  strategies  of  evaluating  community  building  in  R52 1 and  R5 1 1 , although 
holistic  in  spirit,  merely  as  a framework  on  which  more  specific  and  precise  assessments  can  be 
constructed.  We  believe  deeper  exploration  of  success  factors  in  fostering  online  community  would  be  a 
very  fertile  topic  for  further  research  and  warrants  further  investigation. 

Questions  for  Further  Research 

Beyond  the  evaluation  of  the  success  or  failure  of  community  in  the  Fall  2000  Distance  Masters 
Core,  there  are  other  topics  worthy  of  research. 

• What  are  some  valid  measures  of  community  development? 

• If  community  formed,  what  was  its  effect  on  the  learning? 

• How  can  learners  be  motivated  to  take  part  in  virtual  academic  or  social  community  activities? 

• What  are  special  features  of  “forced  community”  like  the  Masters  cohort? 

• What  is  the  expected/observed  life  cycle  of  the  Distance  Masters  learning  community? 

• How  does  this  community  develop  and  maintain  its  history? 

• Should  the  Distance  community  be  integrated  with  the  residential  graduate  community?  If  so,  in  both 
academic  and  social  ways?  If  so,  how  can  this  be  accomplished? 

• How  can  the  community  best  be  mentored? 

• What  are  the  different  roles  for  instructors,  graduate  assistants,  volunteers,  upper-year  1ST  students, 
etc? 

• What  communication?collaboration  tools  foster  the  development  of  a learning  community? 

• What  are  the  best  practices  for  using  existing  communication  tools  in  distance  education? 

• What  tool  features  lend  themselves  to  different  aspects  of  collaboration  and  community-building? 

• How  appropriate  were  the  tools  chosen  for  Fall  2000  in  terms  of  collaboration  and  community 
formation? 

Conclusion 

Having  determined  that  richer  learning  takes  place  within  the  context  of  a learning  community, 
this  report  provides  background  descriptions  of  characteristics  of  community  and,  more  specifically,  a 
virtual  learning  community.  We  discuss  the  goal  of  moving  a cohort  to  a learning  community  through 
scaffolding  activities  rooted  in  the  communication  formats  of  discussion,  cooperation,  and  collaboration. 

The  report  then  treats  the  Core  classes  in  three  separate  sections:  Core  (principally  orientation  and 
the  online  cafe),  R5 1 1 , and  R52 1/522.  The  courses  are  described,  instructional  strategies  and  rationales  are 
presented,  possible  assignments  are  detailed,  and  an  instructor  checklist  is  provided. 

Finally  we  thought  it  necessary  to  determine  some  strategies  to  evaluate  a)  whether  community  has  formed 
within  the  cohort,  and  b)  in  what  ways  the  community  contributed  to  deeper  learning.  We  also  provide 
some  possible  topics  for  further  study. 
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Abstract 

The  purpose  of  this  paper  is  to  provide  an  overview  of  what  steps  were  taken  to  create  an 
electronic  performance  support  system  (EPSS)  prototype  for  Instructional  Systems  Technology 
(1ST)  doctoral  students  to  help  them  prepare  for  the  Qualifying  exams . Throughout  the  design  and 
development  process  of  the  EPSS,  the  principles  of  socio-technical  design  and  knowledge 
management  were  used.  To  allow  for  a participatory  design,  we  used  a qualitative  approach  to 
collect  data , and  this  was  followed  by  interactive  designing  of  the  prototype.  We  used  rapid 
prototyping  and  usability  testing  so  that  the  prototype  would  be  customer  driven  and  meet  the 
needs  of  the  end  users. 

1.  Introduction 

The  project  described  in  this  paper  was  organized  to  build  a supportive  environment  for  1ST  doctoral 
students  at  Indiana  University -Bloomington  to  help  them  prepare  for  the  Qualifying  exams.  We  seek  to 
improve  productivity  by  honing  the  skills  and  tools  they  need  and  providing  a supportive  environment 
where  students  can  collaborate  and  seek  advice  from  each  other  during  preparation . During  the  process, 
the  concepts  of  information,  collaboration  technology,  learning  and  work  style,  tools  and  management 
became  important  in  building  the  support  system.  The  perspective  of  socio-technical  design,  knowledge 
management,  and  human  approach  were  applied  throughout  the  process. 

This  paper  covers  the  process  of  designing  this  system.  With  reference  to  the  screen  shots  and  the 
reflections  to  the  design  challenges,  we  hope  to  instigate  readers  to  continue  from  where  we  have  left  off 
and/or  to  explore  future  developments  on  EPSS  design/creation. 

2.  Design  Problem 

This  section  of  the  paper  presents  a basic  description  of  the  1ST  doctoral  qualifying  exam  process.  It 
then  discusses  some  basic  features  of  EPSS.  Finally  it  discusses  the  semester’s  design  challenge1:  to  build 
an  EPSS  to  support  doctoral  students  preparing  for  the  qualifying  exams. 

2.1  The  Qualifying  Exam  Process 

Qualifying  examinations  are  an  important  milestone  for  doctoral  students  in  almost  any  graduate 
program.  In  the  1ST  program,  doctoral  students  must  pass  a written  qualifying  exam  as  well  as  an  oral 
qualifying  exam  before  they  are  admitted  as  official  doctoral  candidates  and  allowed  to  proceed  with  their 
dissertations.  Many  students  view  preparing  for  the  qualifying  exams  as  a hurdle  that  they  must  get  over 
before  moving  on  to  more  meaningful  research/work  as  a graduate  student.  It  is  a process,  which 
traditionally  causes  a lot  of  stress  for  the  individuals  preparing  for  the  exams. 

Recently,  in  the  1ST  department,  the  qualifying  exam  process  changed.  In  the  past,  the  qualifying 
exam  was  based  on  a closed  book  format.  Over  two  four-hour  sessions  on  consecutive  days,  the  students 
had  to  answer  a series  of  short  essay  questions  with  the  purpose  to  be  able  to  analyze  and  synthesize  their 
knowledge  of  the  field  of  instructional  systems  technology.  The  new  process,  called  the  “authentic  Quals” 
gives  students  a period  of  three  weeks  to  write  three  documents  on  given  topics.  The  documents  are  not 


This  is  a one  year  on  going  project.  Paper  was  written  primarily  based  on  the  first  version  of  EPSS  we  created. 


essays  but  rather  formats  (such  as  grant  proposals,  journal  articles,  etc.)  which  resemble  the  types  of  writing 
that  doctoral  students  will  be  engaged  in  as  researchers. 

2.2  Electronic  Performance  Support  Systems 

While  there  is  still  some  debate  regarding  the  definition  and  function  of  electronic  performance 
support  systems  (EPSS),  the  term  generally  refers  to  some  type  of  electronic  integrated  system  or 
infrastructure  which  provides  access  to  information  and  tools  which  enable  individuals  to  achieve  a high 
level  of  performance.  Gery  (1991)  explains  that  an  EPSS  is  custom  designed  to  provide  access  to 
information,  learning  activities,  and  expert  consultations  at  the  moment  of  need.  Brown  (1 996)  explains  the 
role  of  an  EPSS:  “[An  EPSS  provides]  a context  within  which  work  is  done.  Everything  needed  to  do  the 
job— information,  software,  expect  advice  and  guidance,  and  learning  experience — is  integrated  and 

minimal  support  and  intervention  by  others ” He  adds  that  the  concept  of  the  EPSS  is  a shift  away  from 

viewing  workers/performers  as  “people  to  be  trained”  to  viewing  workers  as  “people  who  need  support  on 
the  job.”  An  EPSS  concentrates  on  the  performer,  the  job,  and  job  tasks. 

Corporations  are  benefiting  from  the  implementation  and  usage  of  integrated  electronic  systems 
(Borwn,  1996;  Raybould,  1995).  For  example,  a large  corporation  may  combine  its  many  electronic  tools 
(including  databases,  word  processing,  e-mail,  calendars  etc.)  to  facilitate  easy  access.  This  can  increases 
the  worker’s  efficiency  (Gery,  1991 ; Kavat,  1997;  Raybould,  1995).  In  addition,  an  EPSS  can  also  increase 
the  efficiency  and  productivity  of  a worker  by  providing  just-in-time  coaching  and  tutorials  that  can  reduce 
the  learning  curve  and  improve  performance.  For  example,  a manager  needs  to  write  a performance 
appraisal  for  an  employee.  An  EPSS  can  provide  this  manager  the  employee’s  profile  and  work 
performance,  give  the  manager  advise  messages  of  how  to  review  the  employee’s  performance,  and 
provide  a short  learning  module  on  how  to  write  performance  appraisals. 

2.3  The  Quals  EPSS 

The  design  problem  we  were  faced  with  was  to  design  an  EPSS  that  would  support  doctoral  students 
in  their  preparation  for  taking  the  qualifying  exams.  Although  EPSS  development  and  use  are  more 
commonly  found  in  the  world  of  business,  educators  have  begun  to  explore  EPSS  applications  as  well 
(Chiero,  1996).  More  recently,  integrated  electronic  systems  (EPSS)  have  been  developed  for  use  by 
teachers  in  grades  K-12  (Kavat,  1997)  as  well  as  for  special  education  teachers  in  behavior  management 
(Hung,  1998).  For  example,  an  EPSS  can  provide  assistance  with  the  task  of  creating  an  individualized 
lesson  plan  for  a student  assessed  with  learning  difficulties  in  a particular  subject.  An  EPSS  can  help 
teachers  establish,  maintain,  and  facilitate  an  effective  work  environment  by  improving  the  performance  of 
their  daily  tasks.  Based  on  the  success  of  this  integrated  model,  this  study  is  important  in  discovering  the 
need  and  benefits  of  developing  a similar  system  to  help  doctoral  students  improve  the  efficiency  of  their 
work/study  performance. 

We  believe  that  using  an  EPSS  design  for  the  performance  support  of  qualifying  exam  takers  is  a 
good  match.  The  main  characteristics  of  doctoral  students’  work  for  Quals  are  to  master  the  skills  of 
developing  their  own  ideas  once  they  have  located,  organized,  evaluated,  and  synthesized  the  existing 
literature.  This  is  similar  to  the  work  in  many  large  corporations  and  educational  settings  that  have 
benefited  from  EPSSs  in  the  past.  Also  the  tasks/works  that  relate  to  Quals  preparation  involve  vast 
amounts  of  knowledge,  technology,  and  high  performance  expectations.  These  features  are  a few  of  the 
major  reasons  that  Brown  (1996)  mentioned  in  outlining  when  an  organization/institution  should  choose  an 
EPSS  solution. 

3.  Design  Process 

3.  1 Needs  Analysis 

In  order  to  design  a product  that  will  enable  people  to  work  effeciently,  we  must  define  and 
understand  how  they  work  and  what  they  need  to  do  their  jobs,  as  well  as  the  whole  environment.  We  feel 
that  conducting  a complete  and  detailed  analysis  before  we  begin  designing  it  is  premature.  The  gradual 
determination  of  the  finer  details  of  analysis  will  emerge  as  the  various  levels  of  users  interact  with  a 
demonstration  prototype.  At  this  stage,  the  focus  of  the  analysis  is  to  list  the  performed  tasks,  the  process  of 
detailing  work  flow,  and  the  process  and  skills.  The  analysis  techniques  include  interviews,  observation, 
and  document  analysis. 
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3.1.1  Participants 


We  collected  data  from  people  who  are  involved  in  the  quals  process,  including  faculty  members,  the 
quals  committee,  the  department  chair,  and  doctoral  students.  Shown  in  table  1,  the  main  charactersitics  of 
doctoral  students  that  we  considered  based  on  the  deparment  profile  are  gender,  maritalstatus,  work  status, 
transfer  students,  and  quals  taken. 


Table  1 . Characteristics  of  Participant 


Quals 

Taken 

Transfer 

Student2 

Work  Status 

Marriage 

Status 

Gender 

Yes 

No 

Yes 

No 

P/T 

F/T 

None 

M 

S 

M 

F 

6 

7 

8 

5 

12 

1 

0 

6 

7 

6 

7 

3.1.2  Multiple-Perspective  Anslysis 

During  the  process,  detailed  tasks,  workflow,  the  process  and  skills  were  analyzed  from  socio- 
technical  and  user-centered  desgin  perspectives.  We  determined  how  each  task  component  contributes  to 
the  product  based  on  Carr’s  classification  of  tasks:  accomplishments,  ancillary  activities,  and  distractions 
(1992.)  This  analysis  helped  us  to  derive  efficiencies  from  electronic  support.  The  effeciency  is  increased 
by  removing  distractions,  and  effectiveness  is  increased  by  supporting  completion  of  necessary  ancillary 
tasks  and  accomplishments. 

4.  Designing  Principles  and  Rapid  Prototyping 

Major  principles  other  than  user-centered  design  principles  for  multi-media,  computer  system  design, 
goal — to  reduce  stress,  and  change  behavior  pattern. 

With  the  analysis  framework  and  all  the  behavior  patterns  we  found,  16  categories  of  needs  were 
emerged.  They  are: 

• Information 

• Tools; 

• Stress  reduction 

• Cooperation  & collaboration 

• Access  to  resources/information 

• Suggestions  and  tips 

• Writing  strategies 

• Clear  expectations 

• Language  needs 

• Physical  organization  readings 

• Individual  work 

• Reading  strategies 

• Useful  courses 

• Time  management 

• Extenuating  circumstances 

• Location  for  study  & writing 

Then  we  created  and  designed  “EPSS  solutions”  which  are  suggested  for  each  “need”  and  would 
emerged  in  various  levels  in  the  system.  Again,  a socio-technical  approach  was  applied  through  out  the 
whole  process.  We  discussed  each  item  (task,  process,  need,  and  intervention,  etc.)  back  and  forth  between 
micro-level  and  macro-level  and  among  those  perspectives  and  design  guidelines  we  mentioned  before. 
Figure  3 was  part  of  the  instrument  that  we  used  to  facilitate  our  discussion.  It  shows  a partial  log  of  the 
discussion.  The  category  of  need  could  be  checked  for  more  than  one  socio-technical  area:  Work  practice, 
people,  environment  technology. 


2 Transfer  student  indicates  the  students  who  have  been  through  the  “core  course”  that  is  required  in  the 
first  year  of  the  1ST  master  program. 
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Figure  3:  Example  of  application  of  socio-technical  principles  to  writing  strategies 

By  identifying  needs  and  coming  out 
intervention  within  border  pictures  and  among 
all  possible  interaction  that  we  could  think  of, 
the  major  conceptual  categories  that  we  would 
like  to  support  are: 

(a)  information  management 

(b)  stress  management 

(c)  collaboration  management; 

(d)  productive  through  embedded 
guidance,  advice,  feedback  and 
work  metaphors;  and 

(e)  a problem-solving  environment 

that  integrates  basic  tools, 
information  management, 

collaboration  and  productivi  ty  tools 
in  a seamless  environment. 

The  EPSS  included  conceptual  tools  such  as  information  managers,  collaborative  utilities,  and 
guidance  (including  suggestions,  advices,  war  stories,  reflections)  and  support  mechanisms  that  reflected  or 
embodied  “expert  behaviors,”  that  we  thought  are  keys  that  can  lead  to  good  performance  in  the  qualifying 
exams  and  should  be  developed  for  the  quals.  They  were  identified  from  the  interviews  with  those  who  had 
successfully  passed  the  qualifying  exams3  and  faculty  members. 

Building  around  a strong,  meaningful  metaphor  is  key  in  our  design.  We  tried  to  incorporate  a 
meaningful  metaphor  into  EPSS  to  allow  students  envision  the  process  at  the  early  stage  and  to  alleviate  the 
stress  that  students  felt  during  the  preparation  of  the  exams.  Hoping  this  strategy  has  impact  on  their 
behavior,  such  as  start  early,  clear  vision,  reduce  stress,...  etc..  We  would  like  our  potential  users  to  actively 
participate  and  buy  into  the  EPSS  program. 

5.  EPSS  Prototype  and  Knowledge  Management 

In  this  general  overview  section  we  explain  several  strategies  that  we  tried  to  use  the  EPSS  to  address 
these  issues.  Some  concepts  learned  from  knowledge  management  literature  that  has  been  incorporated  into 
the  design  will  be  discussed  as  well. 

5.1  General  Design  Overview 

Two  of  the  main  issues  that  we  wanted  to  address  with  the  Quals  EPSS  are: 

■ reducing  the  stress  involved  in  Quals  process 

■ making  the  Quals  preparation  process  more  meaningful  to  the  professional  development  of 
the  students 

One  thing  that  we  wanted  the  EPSS  to  do  was  to  be  a catalyst  for  changing  the  way  students 
perceived  the  Quals  process . We  felt  that  it  was  important  to  help  students  get  the  bigger  picture  early  on 
about  how  the  qualifying  exams  fit  into  the  whole  doctoral  program  and  their  future  professional 
development.  We  hope  to  help  students  perceive  the  qualifying  exam  process  as  a learning  process  that  is 
just  an  extension  of  the  other  professional  activities  they  are  and  will  be  involved  in. 

5.1.1  The  Bigger  Picture 


Category:  C2  Writing  Strategies 


Work 

Practice 


People 


Environment 


Technology 


x 


EPSS  Suggestions 


1.  online  debates  around  specific  questions 

(KNOWLEDGE  SHARING) 

2.  connect  to  R7 1 1 area 

3.  examples  and  tutorials  for  each  of  the  writing 
formats  (PERFORMANCE) 

4.  collaboration  with  R711  - suggestion  to  take 
(KNOWLEDGE  SHARING) 

5.  reference  books  about  how  to  improve  writing. 
(INFORMATIONAL) 

6.  contact  information  for  writing  labs 
(INFORMATIONAL) 


3 


Mistakes  made  by  all  takers  also  played  an  important  role  to  help  us  identify  key  behaviors. 
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In  order  to  provide  a general  EPSS  tool  that  would  be  useful  to  all  students  in  the  1ST  program,  the 
Quals  EPSS  was  designed  as  a part  of  a larger  EPSS.  This  EPSS  would  allow  students  to  customize  their 
profiles  and  include  components  such  as  the  Quals  components  if  they  so  desired. 

5.1.2  Timeline  Metaphor 

While  the  mockup  in  Figure  3 shows  the  top  level  view  based  on  profiles,  we  also  thought  that  the 
use  of  a timeline  metaphor  for  the  doctoral  student  profile  would  be  useful.  Figure  3 shows  a screen  shot  of 
how  this  metaphor  might  be  integrated  into  the  EPSS. 

The  use  of  the  timeline  metaphor  has  several  benefits  that  help  us  to  reach  the  goals  of  the  EPSS. 
Primarily,  it  helps  to  place  the  Quals  EPSS  components  into  the  larger  framework  of  getting  through  the 
doctoral  degree  program.  In  the  ideal  situation,  students  would  be  able  to  easily  navigate  back  and  forth 
between  the  different  EPSS  components.  Each  “mini”  EPSS  would  help  support  activities  specific  to 
different  milestones  in  the  doctoral  student  timeline. 

Part  of  the  impetus  for  using  the  timeline  metaphor  was  also  to  expose  incoming  students  early  on  in  their 
program  to  information  and  tools  related  to  the  Quals  process.  We  feel  that  doing  this  is  a step  towards 
helping  students  to  change  their  practices  of  preparing  for  Quals  a semester  before. 


Additionally,  combining  the  components  under  a common  metaphor  allows  for  emphasis  of  skills  and 
other  commonalities  between  major  doctoral  program  milestones. 

Finally,  wthin  the  Quals  EPSS  components,  we  continue  using  the  timeline  metaphor.  Figure  3 
shows  how  the  user  interface  emphasizes  the  steps  that  need  to  be  taken  in  order  to  complete  the  qualifying 
exam  process.  Prominently  featured  in  the  design  is  the  acquisition  of  skills. 

5.1.3  Skills  Development  Orientation 

Another  goal  of  our  design  was  to  not  only  reduce  the  stress  level  of  studying  but  to  also  make  the 
Quals  process  more  meaningful  to  the  doctoral  students.  The  department  has  taken  large  strides  in  this 
direction  by  moving  to  the  “authentic”  Quals  format.  We  have  tried  to  take  it  a step  further  by  highlighting 
within  the  EPSS  three  specific  skills  that  will  help  students  to  pass  the  qualifying  exams.  The  skills  are: 

• Reading  Skills 

• Writing  Skills 

• Research  Skills 

Highlighting  these  three  skills  is  key  to  the  EPSS  because  these  are  the  same  skills  that  a 
student  must  acquire  in  order  to  publish,  to  get  through  the  dissertation  process,  and  to  survive 
professionally. 
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5.2  Tacit  vs.  Explicit  Knowledge  Features 

Tacit  knowledge  includes  ideas  that  are  created  in  the  mind  of  the  individual.  It  also  includes  the 
“mental  model”  or  a framework  of  how  we  see  and  perceive  the  world.  Working  mental  models  can  include 
“schemas,  paradigms,  beliefs,  and  viewpoints**  (Nonaka,  1994,  p.16).  Tacit  knowledge  is  deeply  rooted  in 
the  way  we  behave  and  think.  As  a result,  an  effort  must  be  made  to  draw  it  out  of  individuals  and  make  it 
available  to  others.  Conversely,  explicit  knowledge  refers  to  knowledge  that  is  “transmittable  in  formal 
systematic  language”  (Nonaka,  1994,  p.16).  Hence  explicit  knowledge  may  be  more  readily  available  to 
others.  We  have  tried  to  apply  these  principles  of  knowledge  management  into  our  design  prototype. 

5.2.1  Transferring  Tacit  Knowledge 

As  knowledge  is  not  static,  but  rather  dynamic,  Nonaka  refers  to  several  kinds  of  knowledge 
conversion:  Tacit  to  tacit,  Tacit  to  Explicit,  Explicit  to  Explicit  and  Explict  to  Tacit.  Tacit  to  tacit 
knowledge  creation  (labeled  socialization  by  Nonaka)  creates  types  of  knowledge  such  as  embedded  and 
encultured  knowledge  (Blackler)  that  tend  to  be  found  at  more  of  an  organizational  level.  These  types  of 
knowledge  depend  heavily  on  shared  understandings  to  be  created.  They  can  also  reside  in  systemic 
routines  and  cultures  found  within  organizations  (see  Figure  6).  We  tried  to  incorporate  encultured 
knowledge  by  providing  scenarios  where  the  user  can  read  about  or  listen  to  the  process  of  how  one  student 
explains  to  another  what  steps  they  took  to  prepare  for  the  Quals  and  what  things  they  would  have  done 
differently.  It  is  explained  in  terms  of  the  1ST  culture  and  how  work  gets  done  in  1ST.  The  process  of 
socialization  is  one  that  may  be  the  most  difficult  to  facilitate  with  an  EPSS;  we  felt  that  the  a system  in 
which  people  with  a shared  goal  of  passing  the  qualifying  exams  can  meet  is  one  of  the  first  steps  to 
creating  a community.  We  provided  students  a social  space  where  students  can  learn  in  a collaborative 
effort.  In  each  section  of  Reading,  Writing  and  Research  skills,  students  can  discuss  how  they  would  like  to 
share  their  exemplar  pieces,  or  seek  input  from  each  other  of  how  they  would  answer  a question.  In 
addition,  there  is  a space  for  them  to  discuss  personal  issues  such  as  coping  with  stress.  It  may  be  that  just 
having  the  system  in  place  will  provide  people  with  the  opportunities  to  contact  each  other  and  begin  the 
process  of  sharing  and  creating  tacit  knowledge.  Tacit  to  explicit  knowledge  creation  (labeled 
extemalization  by  Nonaka)  will  most  likely  generate  encoded  knowledge  (Blackler).  Encoded  knowledge  is 
information  that  is  conveyed  through  signs  and  symbols.  This  happens  when  individuals  try  to  codify  or 
physically  represent  some  piece  of  knowledge.  We  found  that  the  knowledge  of  how  to  successfully  pass 
the  qualifying  exams  were  in  the  minds  of  the  few.  This  was  done  by  capturing  war  stories  and 
documenting  students  who  have  taken  the  Quals  and  share  what  they  thought  were  good  and  bad  practices. 

5.2.2  Making  Knowledge  Explicit 

Explicit  to  explicit  knowledge  creation  (labeled  combination  by  Nonaka)  represents  an  area  where 
information  processing  can  create  new  knowledge.  New  encoded  knowledge  can  be  created  through  this 
process  as  well  as  embrained  knowledge.  New  embrained  knowledge  could  be  created  as  knowledge  “that” 
and  knowledge  “about”  expands  due  to  transformations  of  explicit  knowledge.  Through  the  process  of 
combination  we  tried  to  take  explicit  knowledge  that  already  exists  about  the  Quals  and  process  it  into  a 
form  that  is  more  conducive  to  individual  knowledge  acquisition.  This  was  primarily  done  by  having  all  the 
information  about  the  Quals,  the  reading  list  and  being  able  to  practice  skills  for  the  Quals,  all  in  one  place. 
This  also  allows  for  group  interactions  in  a discussion  forum  where  ideas  are  discussed,  transformed,  and 
enriched. 

Explicit  to  tacit  knowledge  creation  (labeled  internalization  by  Nonaka)  represents  what  we  most 
often  consider  to  be  traditional  learning.  This  is  where  knowledge  in  explicit  form  (often  times  abstract 
knowledge)  is  made  tacit  by  putting  the  abstract  knowledge  to  use  in  a real-world  situation  or,  in  other 
words,  learning  through  grounding  the  abstract  in  concrete  situations.  This  move  from  explicit  to  tacit  often 
times  creates  embodied  knowledge  or  knowledge  about  “how”  things  are  done  or  familiarity  or 
acquaintance  with  a system.  The  process  of  converting  explicit  to  tacit  knowledge  can  also  create  new 
levels  of  embrained  knowledge.  The  process  of  internalization  was  included  into  the  design  of  the  EPSS  by 
providing  ways  for  the  students  to  convert  their  explicit  knowledge  into  tacit  knowledge.  For  example, 
there  were  tools  to  help  students  take  a technical  understanding  of  the  writing  formats  and  convert  it  into 
tacit  knowledge  through  practice  and  comparison  with  exemplars. 
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5.3  Quals  EPSS  as  a Productivity  Tool 


One  of  the  findings  from  our  needs  analysis  was  that  doctoral  students  in  general  are  juggling  many 
different  responsibilities  and  are  very  interested  in  improving  the  efficiency  of  their  efforts  especially  when 
it  comes  to  the  qualifying  exam  preparation  process.  For  this  reason  we  believe  that  the  functionality  of  our 
Quals  EPSS  is  more  heavily  weighted  towards  the  productivity  end  of  the  spectrum  than  towards  the 
innovation  end  of  the  spectrum.  Many  of  the  features  of  the  EPSS  (such  as  sharing  advice,  war  stories,  etc.) 
are  geared  towards  capturing  innovative  ideas  from  past  Quals  takers  and  making  them  available  in  an 
explicit  way  to  current  Quals  takers.  The  idea  is  that  we  want  the  EPSS  to  improve  the  productivity  of 
current  Quals  takers  by  making  their  studying  efforts  more  efficient.  Making  appropriate  information 
readily  accessible  to  the  Quals  takers  is  also  a way  that  the  EPSS  enhances  the  productivity  of  its  users.  A 
few  types  of  information  that  are  made  accessible  to  the  users  are: 

• Official  Information 

• Policy  statement 

• Writing  formats 

• Ethical  issues/  code  of  conduct 

• Grading  rubrics 

• Reading  List 

• Library/Research  Resources 

• Accessing  databases  (knowing  which  databases  to  go  to) 

• Keyword  / descriptors,  related  to  1ST  - by  topics 

• Using  bibliographic  info 

• Links  to  on-line  full  text  / e-joumals 

• Links  and  list  of  Core  journals  in  the  field 

• Basic  search  skills  / strategies 

• Bibliographic  Info 

• Basics  on  ProCite  and  EndNotes 

• Writing  format  style  guides  (such  as  APA) 

In  addition  to  providing  needed  information  to  students,  the  EPSS  design  tried  to  also  focus  on 
creating  performance  interventions  that  would  help  to  improve  an  individual’s  productivity.  An  example  of 
one  such  intervention  is  providing  each  student  with  a checklist  of  the  readings  on  the  quals  reading  list. 
The  student  can  then  add  readings  to  the  list,  set  a reading  schedule,  and  keep  track  of  what  he/she  has  read. 
Figure  4 shows  a mock  up  of  how  an  intervention  such  as  this  might  look  in  the  EPSS. 


5.4  Knowledge  Management  Objectives 


Davenport  (1998)  detailed  the  four  following  knowledge  management  objectives. 

• Creating  Knowledge  Repositories 

• Improving  Knowledge  Access 

• Enhancing  the  Knowledge  Environment 

• Managing  Knowledge  as  an  Asset 
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This  section  explains  how  specific  features  of  our  Quals  EPSS  help  to  accomplish  these  different 
objectives. 

5.4.1  Creating  Knowledge  Repositories 

There  are  three  main  features  in  our  design  related  to  creating  knowledge  repositories:  (1)  providing 
static  information,  (2)  providing  dynamic  (updateable  information),  and  (3)  creating  meaningful  categories 
for  locating  appropriate  information. 

5.4. 1.1  Providing  Static  Information 

There  are  lots  of  specific  pieces  of  static  information  that  are  provided  through  the  Quals  EPSS. 
Section  4.3  of  this  report  lists  some  of  them  in  detail.  Static  information  was  typically  provided  as  links  to 
resources  or  official  information.  Figure  8 shows  an  example  of  a screen  shot  of  a part  of  the  EPSS  where 
static  information  regarding  official  Quals  information  is  provided. 

5.4. 1.2  Providing  Dynamic  (updateable)  Information 

There  were  many  ways  in  which  we  tried  to  provide  dynamic  information  to  the  EPSS  user.  One  of 
these  ways  was  to  provide  a space  for  individuals  to  post  questions  and  get  responses.  For  this  we  used  the 
public  folder  feature  of  MS  Outlook.  The  EPSS  contains  a separate  folder  for  each  of  the  main  skill  areas  of 
reading,  writing,  and  research.  Each  of  these  folders  has  a “Questions”  sub-folder  where  students  can  post 
questions  they  have  and  get  answers  posted  by  other  students  or  faculty  members. 

Additionally,  the  EPSS  will  have  forms  that  allow  students  who  have  already  taken  the  exams  to 
submit  advice  and  war  stories  to  a database  for  students  who  will  be  taking  the  exams  in  the  future  (see 
Figure  5). 


Another  dynamic  feature  of  the  EPSS  will  allow  students  to  share  information  about  bibliographic 
databases  as  well  as  the  database  files  themselves.  So  a student  who  wishes  to  use  EndNotes  may  get 
advice  and  give  advice  to  others  about  how  to  use  the  tool.  The  EPSS  will  also  allow  individuals  to  share 
their  EndNotes  or  ProCite  database  files  with  other  students.  So  a student  just  starting  the  program  could 
begin  with  an  EndNotes  database  file  that  already  has  the  qualifying  exam  readings  in  it. 

5. 4. 1.3  Meaningful  Categories  for  locating  information 

A final  area  that  is  related  to  the  EPSS  providing  knowledge  repositories  to  the  students  is  through  its 
organization  of  the  information  into  meaningful  categories.  A knowledge  repository  is  not  very  useful  if 
you  cannot  quickly  find  the  information  you  want  within  it.  In  the  EPSS  we  tried  to  simplify  the  interface 
by  creating  a timeline  of  tasks  related  to  the  Quals  and  also  focusing  on  skills  acquisition  (See  Figure  1 1). 
We  felt  that  most  of  the  critical  information  fit  nicely  into  these  categories.  In  addition  to  providing  a few 
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categories,  a search  capability  is  also  planned  for  the  EPSS,  so  that  the  entire  EPSS  database  can  be 
searched  for  key  words. 

5.4.2  Improving  Knowledge  Access 

Finding  ways  to  provide  improved  access  to  available  knowledge  was  another  focus  of  our  design. 
The  primary  way  we  did  this  i was  to  provide  a mechanism  for  the  sharing  of  contact  information  among 
individuals  who  are  taking  the  qualifying  exams.  Figure  12  shows  an  example  of  the  contact  list,  which  is 
updateable.  Only  students  wishing  to  have  their  names  included  on  the  contact  list  would  be  listed  there.  In 
addition  to  regular  contact  information,  the  contact  list  gives  information  regarding  when  each  individual 
plans  to  take  the  qualifying  exams. 

A few  other  ways  in  which  access  to  information  rather  than  the  information  itself  is  provided 
through  the  EPSS  is  via  access  to  the  Quals  preparation  listserv,  providing  contact  information  for  the 
writing  labs,  and  through  access  to  information  and  writing  feedback  available  to  the  R71 1 students. 

Finally,  access  is  provided  to  library  resources.  However,  we  didn’t  want  to  just  provide  links  to  the 
main  library  resource  page.  So,  under  the  “Research”  area  of  the  EPSS  we  provide  access  to  the  specific 
online  journals  that  are  related  to  our  field.  We  also  provide  information  regarding  listings  and  call  numbers 
for  non-electronic  journals  related  to  1ST  as  well  as  search  terms  related  to  the  field  and  especially  related 
to  the  reading  categories. 

5.4.3  Enhancing  the  Knowledge  Environment 

We  attempt  to  enhance  the  knowledge  environment  by  making  it  comfortable  and  useful  for  everyone 
that  wishes  to  use  it.  We  tried  to  do  this  by  using  the  timeline  metaphor  that  pervades  the  design  prototype. 
We  use  this  metaphor  to  try  and  subtly  facilitate  a behavior  change.  We  hope  that  the  timeline  can  serve  as 
a friendly  reminder  to  students  of  how  their  preparation  fits  into  the  larger  picture. 

We  also  attempt  to  enhance  the  KM  environment  by  including  a scenario  in  the  EPSS.  The  purpose  of 
the  scenario  is  to  get  students  thinking  early  on  about  the  issues  related  to  quals  and  to  get  them  motivated 
to  start  preparing  for  the  quals  early.  In  a way  we  are  using  the  scenario  to  try  and  increase  “cultural 
receptivity”  to  the  quals  preparation  process. 

We  also  believe  that  the  knowledge  environment  is  enhanced  through  the  integration  of  important 
skills  such  as  reading,  writing,  and  research  throughout  the  entire  Quals  EPSS  as  well  as  the  other  EPSS 
components  under  the  doctoral  student  profile  in  the  1ST  EPSS.  The  knowledge  environment  is  enhanced 
because  familiar  themes  in  the  form  of  skills  will  surface  no  matter  where  the  user  is  in  the  system. 

5.4.4  Managing  Knowledge  as  an  Asset 

Figuring  out  how  to  manage  knowledge  as  an  asset  was  one  of  the  most  difficult  challenges  of  the 
EPSS.  There  are  two  main  ways  that  we  see  the  EPSS  playing  a role  in  managing  knowledge  as  an  asset. 

• through  acquisition  of  valuable  skills 

• through  sharing  of  bibliographic  database  files  - combining  assets  of  individuals  to  make  a larger 
whole. 

Skills  such  as  reading,  writing,  and  research  are  certainly  invaluable  assets  to  any  doctoral  student. 
One’s  ability  to  adeptly  perform  tasks  related  to  these  three  areas  is  certainly  a form  of  “intellectual  capital” 
belonging  to  the  student.  The  EPSS  helps  students  to  acquire  and  enhance  these  skills. 

Secondly,  knowledge  can  be  captured  in  bibliographic  database  files  (with  ProCite  or  EndNotes). 
This  might  include  the  bibliographic  information  as  well  as  short  summaries,  key  words,  etc.  Once  this 
knowledge  has  been  captured,  the  EPSS  provides  an  easy  way  for  students  to  manage  and  share  this 
explicit  knowledge  as  an  asset. 

6.  Reflections 


As  we  reflect  back  on  the  process  that  we  have  gone  through  in  rapid  prototyping  an  EPSS  to  support 
doctoral  students  preparing  to  take  the  qualifying  exams,  we  feel  that  we  have  learned  a lot  about  EPSS 
design.  In  particular,  both  user-centered  and  socio-technical  perspectives  and  knowledge  management  have 
impacted  our  design. 
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6.1  Underlying  Psychology 

As  we  began  to  work  on  the  EPSS  using  a socio-technical  design  framework,  we  really  focused  on 
the  “work  practices”,  the  process  of  work  practice,  and  the  interaction  with  environment,  people  and 
technology.  We  then  tried  to  understand  the  underlying  psychology  behind  why  students  do  what  they  do  in 
preparing  for  the  qualifying  exams.  This  focus  helped  us  to  identify  the  components  of  (1)  stress  and  (2) 
desire  for  efficiency  that  seem  to  pervade  most  student’s  approach  to  preparing  for  the  qualifying  exams. 
Once  we  understood  this,  we  were  able  to  develop  the  timeline  metaphor  for  the  EPSS.  Also,  it  became  one 
of  the  frame  factors  that  helped  us  to  focus  many  of  the  interventions  within  the  EPSS  towards  skills 
development  in  the  areas  of  reading,  writing,  and  research. 

6.2.  Knowledge  as  Process  vs.  Knowledge  as  Object 

Another  idea  that  impacted  us  was  changing  our  conception  of  knowledge  from  only  “knowledge  as 
object”  to  also  include  “knowledge  as  process.”  At  the  outset  of  the  project,  our  group  had  the  idea  that  an 
EPSS  was  primarily  a “knowledge  bank”  or  “knowledge  repository”  where  knowledge  objects  were  stored 
and  retrieved  at  appropriate  times.  Fairly  soon  into  the  semester,  we  began  to  change  our  ideas  as  we  were 
exposed  to  the  concept  of  “knowledge  as  process.”  This  drastically  changed  how  we  viewed  the  EPSS. 
Instead  of  just  looking  for  types  of  information  that  we  could  provide  to  individuals,  we  began  to  look  at 
how  our  EPSS  could  actually  act  as  a catalyst  to  change  the  way  people  prepared  for  the  qualifying  exams. 
This  led  us  to  try  and  develop  and  incorporate  more  performance-based  interventions  into  our  EPSS  design. 

7.  Conclusion 

We  applied  a socio-technical  design  framework,  user-centered  perspective,  and  a rapid  prototyping 
approach  to  the  EPSS  design.  The  design  was  developed  by  looking  at  the  work  practices  and 
psychological  behaviors  of  individuals  currently  preparing  to  take  the  qualifying  exams  as  well  as 
individuals  who  have  already  taken  the  qualifying  exams.  The  EPSS  is  built  upon  the  idea  that  knowledge 
can  be  treated  both  as  “object”  or  “process”  and  thus  incorporates  both  informational  interventions  as  well 
as  more  performance-oriented  interventions. 

Two  of  the  main  (interrelated)  goals  of  the  EPSS  design  have  been  to  (1)  reduce  the  stress  of  those 
preparing  for  the  qualifying  exams  and  (2)  to  increase  the  efficiency  with  which  the  students  can  prepare 
for  the  exams.  These  goals  are  achieved,  at  least  in  part,  by  providing  an  environment  which  encourages 
students  to  begin  preparing  earlier  than  a semester  before  the  quals  by  working  to  develop  skills  in  the  area 
of  reading,  writing,  and  research  that  will  be  useful  throughout  the  program  as  well  as  in  their  professional 
activities.  They  are  achieved  by  providing  the  environment  that  allows  students  to  develop  problem-solving 
skills  in  the  areas  of  information  management,  stress  management,  collaboration  management,  and 
productivity  improvement. 
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Abstract 

The  purpose  of  this  paper  is  to  describe  the  process  of  adaptation  of  R522:  Instructional  Design 
and  Development,  from  a residential  course  to  a web-based  course  offered  in  the  Distance 
Masters  program  in  Instructional  Systems  Technology’  at  Indiana  University . A description  of  the 
web-based  course,  factor  impacting  the  adaption  process , and  recommendations  for  adapting  a 
course  to  the  web  are  also  discussed. 

In  Fall  2000,  the  Instructional  Systems  Technology  department  at  Indiana  University  in 
Bloomington  began  offering  its  36-hour  Master’s’  degree  program  at  a distance.  Half  of  the  1 7 students 
who  enrolled  in  the  program  are  from  Indiana;  the  rest  are  from  out  of  state,  spanning  three  time  zones.  Just 
over  half  of  the  students  are  women;  the  group  is  evenly  split  among  those  who  work  in  K- 1 2,  those  in 
higher  education,  and  those  in  the  corporate  field.  The  projected  time  to  complete  the  program  being  3 
years,  and  during  the  first  semester  of  the  program,  the  students  enrolled  in  four  hours  of  coursework; 
R52 1 : a one-hour  orientation  to  the  field,  and  R522:  a three-hour  course  about  basic  instructional  design 
and  development. 

R522:  Instructional  Design  and  Development  is  the  first  course  the  core  set  in  the  1ST  department. 
Students  in  the  regular  residential  program  come  in  as  a group  and  go  through  a core  set  of  courses;  two  in 
the  Fall  semester  and  two  in  the  Spring.  The  other  usual  Fall  course  is  R5 1 1 : Instructional  Technology 
Foundations  I,  an  introduction  to  and  history  of  the  field.  The  students  tend  to  take  the  core  set  together 
throughout  the  first  year  of  the  program,  after  which  they  have  more  traditional  individual  choice  of  courses 
for  the  rest  of  the  program. 

R522  is  an  in-depth  explanation  of  instructional  design;  the  students  are  introduced  to  the  ADDIE 
model  and  instructional  design  theories,  and  also  work  on  usability,  formative  evaluation,  and  visual 
design.  The  deliverables  for  the  course  include  two  projects;  students  develop  materials  to  teach  one 
concept  and  one  procedure.  Therefore,  they  become  familiar  with  at  least  two  different  instructional  design 
theories  for  each  type  of  instruction,  and  follow  the  procedures  for  materials  design  and  development  for 
two  projects. 

The  1ST  program  focuses  heavily  on  group  work;  students  enter  the  program  and  immediately 
begin  working  very  intensively  with  teams.  The  program  is  also  project-based;  students  work  in  theirteams 
on  two  large  projects  throughout  the  semester.  We  wanted  to  maintain  this  approach  in  the  online  Master’s 
program;  the  ideas  of  intensive  group  work  and  thorough  materials  production  at  a distance  were  the  first 
few  challenges  of  designing  this  course.  Our  program  also  emphasizes  reflective  practice;  students  not  only 
on  look  at  their  finished  product,  but  are  also  required  to  reflect  and  write  about  the  group  process,  group 
dynamics,  team  functioning,  as  well  as  on  the  instructional  design  process  itself.  Student  write  weekly  e- 
mail  updates  throughout  the  semester,  as  well  as  a project  report  at  the  end  that  discusses  the  design 
process;  there  is  as  much  emphasis  on  process  as  on  product.  Finally,  we  wanted  to  maintain  a feeling  of 
community  among  the  students.  As  alluded  to  earlier,  about  50  students,  both  masters-  and  doctoral -level, 
enter  the  program  every  fall . These  students  go  through  core  together  and  really  start  to  develop  a sense  of 
community;  they  feel  like  they’re  all  going  through  boot  camp  together.  This  feeling  of  community  and 
interdependence  is  another  thing  that  we  wanted  to  replicate  in  the  Distance  Master’s  program. 

In  the  residential  program,  incoming  students  have  a one-day  orientation  session  the  week  before 
school  starts.  They  meet  each  other  and  the  faculty,  have  tours  of  the  department  and  the  School  of 
Education,  and  have  a general  introduction  to  the  program.  They  spend  the  next  number  ofweeks  getting  to 
know  each  other  in  classes  and  in  the  social  spaces  in  the  building.  The  Distance  Master’s  students  would 
not  have  this  latter  opportunity,  so  we  created  a 4-day  on-site  orientation  program  for  them.  The  week 
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before  the  beginning  of  the  semester,  all  of  the  distance  students  came  to  Bloomington  to  engage  in  a 
variety  of  content-based,  group-building,  and  administrative  tasks.  We  wanted  to  instill  a “boot  camp”  feel 
to  the  orientation  to  replicate  what  the  residential  students  go  through  in  their  first  few  weeks  of  core.  We 
decided  that  if  we  were  serious  about  the  community  aspect  of  the  program,  we  would  have  to  emphasize  it 
from  the  beginning,  and  that  would  include  bringing  students  and  faculty  together  face  to  face. 

The  problem  that  we  were  trying  to  address  in  designing  a web-based  version  of  R522  was  how  to 
create  an  online  course  that  matches  the  residential  experience  in  terms  of  assignments,  criteria,  the 
problem-  and  project-based  nature  of  the  program,  the  emphasis  on  teamwork  and  group-based 
interactivity,  the  implementation  of  reflective  practice  at  several  levels,  and  maintenance  of  a community  of 
learners  and  cohort  support. 

The  purpose  of  this  presentation  is  to  describe  what  we  went  through  in  the  planning, 
development,  and  implementation  of  a flexible  online  course  based  on  ourprinciples,  which  we  have  taken 
to  be  learner-centered  design,  knowledge  construction,  active  learning,  collaboration,  and  multiplicity. 

Lessons  from  the  literature 

Learners  thrive  in  learning  communities.  We  know  in  the  distance  situation,  students  who  are 
geographically  remote  from  the  instructor  and  other  learners  can  feel  a sense  of  isolation.  One  way  of 
reducing  the  feelings  of  isolation  is  to  know  that  you  have  a number  of  people  you  can  count  on  and  who 
are  doing  the  same  things  and  going  through  the  same  things  on  the  other  side  of  the  country.  Being  part  of 
a community  reduces  attrition.  It  is  well-known  that  when  people  feel  part  of  something,  they  are  less  likely 
to  drop  out  because  they  know  that  someone  else  is  depending  on  them.  If  a student  is  working  on  a group 
project,  then  suddenly  decides  she  really  doesn’t  feel  like  continuing  with  the  program,  what  keeps  her 
engaged  is  knowing  that  others  are  counting  on  her.  The  more  that  learners  need  each  other,  the  better  we 
are  in  terms  of  keeping  them  in  the  program  (Palloff  & Pratt,  1999;  Raymond,  1999). 

Learners  in  an  online  format  need  a great  deal  of  curricular,  instructional,  and  technological 
support  (Sewart,  1 993;  Morgan  & Tam,  1 999).  All  of  these  are  important  in  any  course,  and  at  a distance, 
their  needs  are  magnified  with  regards  to  the  technological  support.  A number  of  presentations  at  this 
conference  were  about  a certain  online  course,  but  students  came  to  campus  anyway.  This  is  not  the  case 
for  us;  since  half  our  students  are  out  of  state,  as  far  away  as  South  Dakota  and  New  York,  coming  to 
campus  is  not  an  option.  If  these  people  could  come  to  Bloomington,  they  would  enroll  in  the  residential 
program.  But  they  can’t,  which  makes  them  extremely  dependent  on  technological  support.  It  is  critical 
then,  that  these  students  be  able  to  access  to  the  kind  of  supports  they  need  in  order  to  achieve  the  goals  of 
the  course,  whether  those  supports  are  library  readings,  grading  rubrics  for  assignments,  presentation 
materials,  or  technological  tools. 

Learners  need  frequent  and  meaningful  interaction  with  the  instructor.  It  also  goes  without  saying 
that,  in  any  course,  learners  seek  interaction  with  the  instructor.  In  a distance  format,  when  students  don’t 
have  the  luxury  of  coming  to  class  every  week  and  seeing  the  instructor,  they  have  to  fulfill  their  interaction 
needs  in  other  ways  (Kirby,  1 999;  Crouch  & Montecino,  1 997).  We  remained  cognizant  of  these 
differences  and  worked  to  build  in  other  opportunities  not  only  for  learner-instructor  interaction,  but  also 
for  learner-learner  interaction. 

Activities  should  be  task-based.  Because  students  who  take  online  programs  are  often  working 
fulltime,  perhaps  even  in  the  field,  they  tend  to  favor  the  practical  over  the  theoretical.  Therefore,  the 
projects  must  be  not  only  applicable,  but  authentic  and  meaningful  as  well.  Evaluation  therefore  should  be 
authentic  and  based  on  those  tasks  (Nelson,  1998). 

Learners  should  be  actively  involved  in  the  evaluation  and  improvement  of  the  course  (Cheung, 

1 998).  Our  students,  being  the  first  cohort  to  go  through  the  program,  know  they  are  working  with  a 
double-edge  sword.  On  the  one  hand,  they  have  the  opportunity  to  almost  mold  the  class  to  their  needs.  On 
the  other,  they  are  working  with  a fledgling  program  that  is  still  working  out  all  its  kinks.  Later  in  this 
paper  we  will  describe  the  dialogue  we  have  with  the  students  about  course  improvements. 

The  final  major  theme  we  found  in  the  literature  was  that  of  faculty  support.  It  is  important  to  offer 
not  only  release  time  for  teaching  online  - which  is,  in  general,  much  more  time-consuming  than  teaching 
in  the  regular  classroom,  especially  in  the  case  of  a new  class  — but  also  provide  remuneration  for  any 
course  development  the  instructor  is  involved  in.  Just  like  students,  faculty  will  need  extensive  technical 
support;  professors  who  are  not  instructional  designers  can  also  benefit  from  instructional  consulting  help 
when  working  in  the  new  format  (Saba,  1999;  Schifter,  2000). 
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Description  of  the  web-based  course 

There  are  three  major  components  of  the  R522  course.  First  is  a website  which  is  a one-way 
communication  vehicle  in  which  the  professor  provides  the  students  with  a great  deal  of  information  about 
the  logistics,  the  topics,  and  the  requirements  of  the  course.  All  of  the  instructor-produced  content  resides 
on  the  web  site,  including  links  to  other  resources  and  information  about  books  and  other  references.  The 
second  part  is  an  asynchronous  discussion  forum  in  which  students  engage  in  online  synchronous  and 
asynchronous  conversation.  They  can  work  at  a class  level,  posting  messages  of  interest  to  all  the  students; 
they  also  have  private  team  areas  that  are  open  only  to  team  members  and  that  other  students  and  the 
instructor  cannot  view.  In  addition  to  posting  and  replying  to  messages,  in  any  of  these  areas  students  can 
post  URLs  for  others’  reference  and  can  upload  documents,  such  as  their  project  reports  or  the  instructional 
materials  they  are  creating.  The  final  element  in  the  course  is  an  e-mail  listserv  through  which  the  instructor 
can  communicate  with  all  students  quickly.  Because  e-mail  is  a “push”  medium  (the  message  arrives  in 
their  inbox),  it  offers  more  immediacy  than  a “pull”  option  like  the  forum  (where  students  have  to  go  to  a 
particular  URL  to  read  a message).  Students  can  also  communicate  with  the  entire  class  via  the  listserv  if 
they  so  desire,  although  thus  far  they  have  very  rarely  taken  this  opportunity.  There  have  been  four 
iterations  of  this  web-site  and  forum  to  this  point  (the  middle  of  the  semester),  so  it  truly  is  an  on-going 
design  process  of  design  and  development  for  the  on-line  resources  of  this  course. 

The  website  is  one-way  information  from  instructor  to  students.  A “home”  link  introduces  the  1ST 
program  and  the  course,  with  each  student’s  digital  picture  there  to  give  a feeling  of  community,  so  that  the 
first  thing  students  see  is  their  classmates  as  a reminder  that  they  are  part  of  the  larger  community.  This 
function  is  helpful  not  only  for  the  students,  but  also  for  the  instructors  and  staff  to  remember  the  students 
they  met  at  the  orientation  and  to  put  a face  to  a name. 

Also  on  the  index  page,  the  “contact  information”  link  lists  how  to  get  in  touch  with  the  instructor, 
the  two  graduate  assistants  who  deal  primarily  with  technological  support,  the  program  coordinator,  and 
department  offices  for  program  and  future  course  questions. 

The  “syllabus”  link  provides  students  with  everything  they  need  to  know  about  how  the  course  is 
run,  including  the  course  philosophy  and  approach,  objectives  for  the  course,  assignments,  evaluation  and 
grading.  Demographic  information  provided  includes  class  meeting  times  and  e-mail  office  hours.  Office 
hours  have  been  set  so  that  students  know  that,  no  matter  when  they  send  e-mails  to  the  instructor,  there  are 
two  times  during  the  week  when  emails  will  be  answered.  This  alleviates  the  concern  that  “I  e-mailed  her 
an  hour  ago,  why  hasn’t  she  answered  me  yet?”  Class  meeting  times  (live  chats)  were  established  as  a 
metaphor  for  a class  meeting:  getting  to  “see”  everyone  at  the  same  time  and  having  the  instructor  there  in 
real  time  to  answer  questions  in  front  of  everyone  and  “asking  in  front  of  the  whole  class”  rather  than 
replicating  private  e-mail  conversations  with  the  professor.  (The  weekly  chat  is  the  only  mechanism  that 
combines  the  two  elements  of  timeliness  and  publicness;  if  you  post  in  the  forum,  you  lose  timeliness  - if 
you  send  private  e-mail,  you  lose  the  publicness.)  Basically,  the  syllabus  is  the  contract  with  the  students 
for  what  they  will  complete  in  the  course. 

The  “schedule”  is  the  driving  page  of  the  website.  This  is  where  the  students  go  in  order  to 
manage  and  work  through  the  course.  The  schedule  provides  a weekly  calendar  of  presentations,  discussion 
topics,  and  the  deliverables  due  for  the  week.  Each  “presentation”  includes  detailed  information  such  as  the 
objectives  of  the  discussion,  an  overview  of  the  key  points  of  the  discussion,  additional  resources  they 
should  read  to  become  more  familiar  with  the  topic,  and  questions  for  reflection.  The  “deliverables”  links 
take  the  students  to  assignment  pages  which  specify  the  date  due,  the  point  value  and  percentage  of  grade, 
any  instructions  for  completing  the  assignment,  and  evaluation  criteria  so  that  students  know  how  the 
assignment  will  be  graded. 

Also  on  the  index  page,  the  students  find  links  to  a “resources”  page  that  provides  a list  of  all 
materials  students  need  to  complete  the  course.  The  “turn  in  your  work”  link  takes  the  students  to  a 
password  protected  fileserver  where  they  upload  their  completed  assignments  to  the  instructor,  so  that  the 
instructor  does  not  have  to  go  search  through  the  forum  or  to  deal  with  e-mail  attachments  every  time  the 
students  hand  in  assignments.  The  “gradebook”  link  is  a feature  provided  through  the  University 
Information  Technology  Services  and  Bureau  of  Evaluative  Studies  and  Testing  (BEST).  Here,  the 
instructor  can  create  a Microsoft  Excel  spreadsheet  with  students’  grades  and  narrative  comments,  and  then 
post  those  to  a password  protected  fileserver  from  which  students  can  access  all  their  grades  and  comments. 
The  index  page  also  features  links  to  an  “evaluation”  website  which  is  also  administered  by  BEST  and 
where  students  complete  midterm  and  final  course  evaluations.  The  index  page  also  includes  links  to  a site 
map,  a “frequently  asked  questions”  page,  and  the  class  Sitescape  Forum. 
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The  second  technological  element  of  the  R522  course  is  the  asynchronous  discussion  forum.  The 
university  has  had  a SiteScape  Forum  license  for  at  least  three  years,  and  this  software  is  used  to  create  a 
virtual  meeting  space  for  the  course,  where  learners  can  discuss  issues  related  to  the  class  with  each  other  in 
both  synchronous  and  asynchronous  formats.  The  R522  class  forum  includes  four  distinct  spaces;  first, 
there  is  a weekly  discussion  topics  space  where  students  go  to  interact  with  the  instructor  and  other  students 
regarding  topics  such  as  analysis,  design,  development,  usability  testing,  creating  instructor  materials, 
implementation,  evaluation,  visual  design,  web  authoring,  concept  learning,  procedure  learning,  group 
skills  for  instructional  design,  and  being  a reflective  practitioner.  A second  space  in  the  class  forum  is 
devoted  to  group  dynamics,  and  students  use  this  space  to  discuss  the  readings  about  group  dynamics  that 
are  required  for  the  course.  Third,  there  is  a general  discussion  area  where  students  can  talk  about  any 
topics  or  issues  related  to  the  course,  but  not  specifically  focused  on  weekly  discussions  or  group  dynamics. 
Finally,  each  student  team  has  a private  space  which  is  password-protected  space  and  where  they  can  go  to 
communicate,  share  documents  and  work  on  their  project  deliverables.  There  is  one  other  SSF  space  that  is, 
strictly  speaking,  at  the  program  level,  not  the  course  level.  The  Core  Cafe  is  a space  dedicated  entirely  to 
social  and  non-R522  topics.  At  the  orientation,  the  students  were  involved  in  designing  a metaphor  (a  town) 
for  this  student-run  space,  and  emphasized  that  it  could  be  used  for  any  non-R522  discussion.  Students 
could  post  messages  about  their  personal  lives  and  jobs,  could  add  links  to  personal  and  career-related  web 
pages,  could  post  hints  for  technical  problems,  etc.  In  the  beginning  there  was  a good  level  of  activity  in  the 
Core  Cafe;  however,  postings  in  the  Core  Cafe  have  dwindled  to  none.  It  is  possible  that  once  the  students 
have  a need  to  communicate  as  a class  (i.e.,  not  just  on  their  project  teams),  about  which  other  courses  to 
take,  etc.,  that  discussion  in  the  Core  Cafe  will  pick  up  again.  It  is  something  that  we  are  actively  watching 
for  research  purposes. 

The  third  technological  element  of  the  course  is  the  class  listserv,  which  is  used  primarily  by  the 
instructor  to  communicate  public  information  that  needs  to  go  to  all  students  at  the  same  time.  Because  of 
the  immediate  nature  of  e-mail,  the  listserv  is  used  for  announcements  that  require  a timeliness  and  priority 
that  would  be  ill-served  by  posting  in  the  SiteScape  Forum.  The  listserv  is  used  by  the  instructor  to  provide 
reminders  of  deliverables  due,  to  make  changes  in  weekly  plans,  the  send  out  clarifications  regarding 
criteria  for  assignments,  etc. 

Instructional  design  process  for  adapting  the  course 

Collaborators  in  course  design 

Our  design  process  was  a collaborative  effort  with  five  stakeholders/groups.  The  client  in  this 
design  project  was  the  instructor  of  the  course,  who  has  a great  deal  of  experience  in  instructional  design 
and  web  development.  Given  this  expertise,  the  client  was  able  to  provide  both  subject  matter  and  technical 
expertise  regarding  the  course  design.  The  Director  of  the  Distance  Master’s  program  was  also  a key 
stakeholder  in  the  design  process,  as  it  was  expected  that  the  design  ofthis  course  might  serve  as  a template 
for  future  1ST  Distance  Master’s  courses.  The  1ST  Department  Chair  was  a third  key  stakeholder  in  the 
design  process,  concerned  with  administrative  aspects  of  the  course  as  well  as  the  relationships  between  the 
Distance  Master’s  course  (R522),  the  Distance  Master’s  program,  the  residential  program  and  school 
requirements.  The  fourth  stakeholder  group,  the  course  designers,  consisted  of  a team  of  five  advanced  1ST 
students,  who  worked  either  as  instructional  or  interface  designers  for  this  project.  These  students  were 
either  enrolled  in  advanced  design  and  development  courses,  or  submitted  the  design  ofthis  course  as  a 
development  project,  which  is  required  for  completion  of  an  1ST  degree.  Finally,  in  addition  to  these  key 
stakeholders,  students  who  had  previously  taken  R522  courses  were  also  involved  in  this  design  project, 
providing  valuable  input  regarding  the  instructional  and  interface  designs  for  the  course  web-site. 
Guiding  principles  for  course  design 

Four  fundamental  principles  guided  all  aspects  of  design  and  development  for  the  R522  course 
website  and  forum.  The  first  principle  was  that  the  web-based  course  must  be  of  comparably  high  quality  to 
the  residential  course;  it  is  not  acceptable  to  have  a “R522  light”  for  distance  students  - or  to  differentiate 
the  quality  of  experience  our  students  received  in  the  course  based  on  the  course  format.  Secondly,  the 
web-based  course  must  serve  as  a model  for  future  web-based  courses  to  be  taught  in  the  1ST  Distance 
Master’s  program,  which  meant  that  the  technologies  used  to  support  the  course  and  the  interfaces  used  to 
present  course  materials  could  not  be  so  highly  specialized  that  they  would  be  difficult  to  replicate  in  other 
courses.  Third,  the  development  process  used  would  have  to  foster  faculty  ownership  and  commitment  to 
the  web-based  course  and  the  Distance  program  because  we  believe  that  faculty  involvement  is  critical  to 
the  overall  success  of  the  Distance  Master’s  program.  Finally,  the  design  team  proceeded  from  the 
assumption  that  an  iterative  model  of  design  and  development  would  be  most  appropriate  for  adapting  the 
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R522  course  from  the  residential  program  to  the  web,  because  the  iterative  approach  would  best  allow 
designers  to  address  the  interactions  between  content  presentation,  instructional  processes  and  interface 
elements,  and  to  adjust  the  course  design  as  appropriate. 

Process  of  adaptation 

The  following  process  was  used  to  adapt  the  residential  course  to  the  web-based  environment 
described  above.  First,  the  design  team  met  with  the  instructor  to  confirm  the  goals  and  objectives  of  the 
course,  and  to  make  sure  that  the  goals  and  objectives  were  similar  to  the  residential  course.  Second,  the 
instructor  identified  the  major  content  components  of  the  course  and  considered  how  these  components 
would  best  fit  in  the  web-based  instructional  environment  (the  major  components  of  the  course  were  the 
instructional  design  process,  instructional  theories  and  group  dynamics  and  critical  reflection  regarding 
instructional  design  and  teamwork).  Having  determined  the  major  components  of  the  course,  the  third  step 
of  adaptation  was  the  establishment  of  an  overall  technology  strategy  for  the  course  (at  this  point,  the 
instructor  and  designers  agreed  on  using  web  pages  to  provide  instructional  materials,  using  the  SiteScape 
Forum  to  foster  group  interaction  and  team  interaction,  and  using  the  listserv  to  communicate  time- 
sensitive  messages  to  the  entire  class).  Having  identified  the  course  components  and  the  technology 
strategy  for  the  course,  the  design  team  worked  with  the  instructor  to  design  an  interface  that  would  best 
reflect  the  priorities  of  the  course  and  emphasize  the  major  course  components.  The  interface  design 
process  involved  the  identification  of  specific  web-pages  and  forum  links  that  would  be  needed,  as  well  as 
deciding  which  elements  would  be  needed  on  each  type  of  page.  The  result  of  this  step  was  the 
development  oftemplates  for  each  element  of  instruction  (templates  for  the  schedule  page,  presentations 
pages,  assignments  pages,  resources,  pages,  etc.).  Once  a list  of  components  had  been  identified,  the 
instructor  and  the  interface  design  team  undertook  discussions  regarding  the  relationships  between  the 
various  web  pages  and  forum  spaces  in  order  to  determine  the  most  appropriate  navigation  paths  and 
navigation  structure  for  the  course  website.  With  templates  for  specific  web-pages  and  a navigation 
structure  in  place,  the  design  team  turned  to  the  task  of  gathering  the  actual  instructional  materials  from 
various  subject-matter  experts,  textbooks,  journals,  web  sites  and  personal  experience  to  write  the  content 
for  the  presentations  pages  and  to  develop  the  other  content  materials  that  were  needed  for  the  web-based 
course,  which  was  step  six  of  the  adaptation  process.  These  content  documents  were  produced  in  Microsoft 
Word  98  and  saved  in  rich  text  format,  so  that  they  could  more  easily  be  imported  to  html  editors  and  coded 
as  html  files,  which  was  the  next  step  of  the  adaptation  process.  After  html  files  were  created,  the  files  were 
usability  tested  with  several  representatives  of  the  target  audience,  identifying  changes  to  be  made  and 
updating  the  pages  based  on  target  audience  feedback.  The  web-site  was  then  ready  to  be  uploaded  to  the 
university  file  server  where  it  was  tested  for  functionality  and  compatibility.  We  learned  that  there  were 
some  compatibility  issues  between  the  Unix  commands  used  by  the  university  servers  and  the 
programming  code  used  when  creating  html  files  with  Microsoft’s  Frontpage  web  editor  software,  resulting 
in  the  need  to  re-code  a number  of  html  files.  During  the  onsite  orientation,  we  showed  the  students  the 
course  web-site  and  trained  them  in  its  use.  At  this  time,  students  also  received  training  on  the  basic 
features  of  the  SiteScape  Forum,  the  group  editing  features  available  in  Microsoft  Word,  as  well  as  basic 
skills  of  teamwork  and  group  dynamics.  We  felt  that  the  course  design  would  not  work  if  we  did  not  train 
the  students  in  how  to  implement  the  design;  and  for  this  reason,  an  orientation  was  designed  not  only  to 
provide  students  with  the  skills  and  knowledge  they  would  need  to  be  successful  in  this  program,  but  also 
to  give  them  the  tools  and  technologies  that  would  support  their  skills  and  knowledge.  Based  on  this 
orientation  experience,  students  began  immediately  to  provide  comments  and  recommendations  for 
improving  the  web  design,  so  a mechanism  was  created  to  gather  their  input,  and  these  inputs  are  used  for 
periodic  maintenance  and  upgrading  of  the  course  website.  We  are  now  engaged  in  ongoing  monitoring  of 
the  university’s  technological  capacity  to  make  sure  that  we  are  continuing  to  provide  our  students  with  a 
course  website  that  is  most  appropriate  to  the  goals  of  instruction  and  best  addresses  students’  needs  given 
available  technologies. 

Factors  impacting  the  adaptation  process 

Given  the  design  process  that  has  been  outlined  above,  the  purpose  of  this  section  is  to  discuss  some 
issues  that  have  probably  impacted  our  development  process,  which  would  need  to  be  taken  into  account  by 
others  who  are  planning  to  adapt  a residential  course  to  the  web  environment. 

Instructor  experience  and  openness  to  innovation 

First,  the  instructor’s  background  and  experience  with  instructional  design  and  with  technology 
impacts  how  easy  or  how  difficult  it  would  be  for  a team  of  designers  to  come  in  and  work  to  create  a 
product  quickly.  In  our  case,  the  instructor  was  a designer  and  an  experienced  computer  user,  so  she  had  a 
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very  good  grasp  of  both  the  process  and  the  product  she  wanted.  Working  with  someone  who  had  varying 
degrees  of  experience  in  pedagogical  strategies,  instructional  design,  and  communications  technology 
would  require  different  things  of  the  design  team  and  would  yield  very  different  results. 

It  is  doubtful  that  an  instructor  would  be  willing  to  teach  a web-based  course  if  that  person  were 
not  in  some  way  open  to  innovation,  yet  instructor  willingness  often  has  little  to  do  with  what  they  end  up 
teaching.  The  degree  of  an  instructor’s  openness,  willingness  to  risk,  and  acceptance  of  ambiguity  will  very 
likely  have  an  impact  upon  how  successful  a web-based  course  may  be. 

Technological  capacity  and  infrastructure 

With  the  development  of  each  course,  there  is  a struggle  to  balance  the  competing  issues  of  what 
technologies  will  be  supported  by  the  university,  what  is  the  greatest  level  of  functionality  that  can  be 
offered,  what  is  the  lowest  common  denominator  of  student  technology  that  must  be  considered,  and  what 
technologies  will  be  most  easily  accessible  and  cost  efficient  for  students. 

Rationale  for  adaptation  to  web-based  instruction 

For  an  1ST  department,  the  creation  of  a web-based  program  works  not  only  at  offering  our 
program  to  those  who  cannot  come  to  Bloomington,  but  also  allows  us  to  practice  what  we  preach.  In  the 
tradition  of  the  old  lab  schools,  it  gives  us  an  opportunity  to  implement  our  ideas,  to  see  if  what  we  are 
reading,  teaching,  and  proposing  for  others  really  does  work,  and  what  the  issues  are.  It  gives  our  students 
an  opportunity  to  be  involved  in  the  design,  development,  and  maintenance  of  a program  before  they  go  out 
into  the  world  to  create  their  own.  The  reasoning  behind  the  creation  of  web-based  instruction  is  likely  to 
impact  the  design  emphasis,  as  well  as  the  time  that  can  be  dedicated  to  the  adaptation  process.  Ifwe  didn’t 
have  students  pushing  us  forthis  type  of  experience  and  needing  to  have  this  type  of  experience,  we  may 
not  have  been  so  quick  to  create  the  Distance  Master’s  program. 

Fit  of  course  goals  with  technological  capacity 

Another  factor  that  impacts  the  adaptation  process  is  the  fit  of  course  goals  with  technological 
capacity.  How  much  of  a project-focus,  readings  focus,  team-based  focus,  and  individual  focus  there  is  in  a 
course  will  determine  what  type  of  web-based  design  is  needed.  For  example,  all  of  the  instruction  and 
interaction  in  R522  is  text-based.  In  the  Spring,  we  will  offer  R54 1 , a production  course  in  which  students 
do  individual  projects  in  Director,  a web  page,  an  audio  presentation,  and  video  presentation.  The  content 
of  that  course  will  greatly  increase  the  technological  needs  and  will  require  different  tools  and  ways  of 
working. 

Labor  force  available  for  course  adaptation 

The  instructor  of  this  course  could  not  have  created  the  course  materials  alone.  Fortunately  we  had 
graduate  students  who  needed  and  wanted  this  experience  and  who  were  willing  to  work  for  course  credit 
or  to  complete  a required  development  project  - if  we  hadn’t  had  that  labor  force,  it  would  have  been  all 
but  impossible  to  get  this  material  developed  in  the  six-week  time  frame  during  which  it  was  completed. 
The  make-up  of  your  labor  force  may  differ:  you  may  need  to  do  much  of  the  instructional  design  on  your 
own  and  delegate  the  web  development  to  others.  Your  time-frame  may  permit  a few  people  to  work  more 
slowly  on  the  development. 

Technological  equipment  and  sophistication  of  learners 

As  the  faculty  member  teaching  this  course,  one  of  the  things  that  I’m  most  appreciative  of  is  that 
the  program  development  team  created  a set  of  minimum  technology  standards  for  the  students,  and  told 
prospective  students  that  they  could  not  enroll  in  the  program  if  they  didn’t  have  technology  that  met  these 
basic  specifications  such  as  processor  speed  and  modem  speed  and  number  of  phone  lines.  We  designed 
with  a certain  expectation  in  mind,  and  even  then,  we  have  made  changes  to  make  sure  that  we  addressing 
the  lowest  common  denominator  of  technology  sophistication  so  that  we  aren’t  leaving  any  of  our  students 
behind.  Nonetheless,  we  think  it  is  not  a burden  to  require  that  Distance  students  have  more  sophisticated 
computer  equipment  and  connectivity  capacities  than  residential  students. 

Learner  motivation  for  taking  the  course 

We  have  been  trying  diligently  in  the  design  to  create  an  environment  in  which  students  really 
want  to  collaborate  with  team  members,  and  to  be  engaged  in  community.  We  continue  to  think  in  terms  of 
the  design  about  how  best  to  address  community.  The  challenge  with  this  type  of  program  is  to  learn  how 
to  balance  the  interests  of  independent,  self-motivated  learners  with  the  need  to  build  professional 
community,  to  match  those  interests  with  the  goals  and  purposes  of  the  program,  and  to  try  to  design  a 
course  website  that  links  the  goals  of  the  learners  with  the  goals  of  the  course. 

Administrative  support 

We  have  two  graduate  assistants  who  are  full-time  technology  support  for  the  course  website,  and 
this  frees  up  the  instructor  to  spend  time  addressing  the  curricular  and  instructional  issues  related  to  the 
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course.  Without  this  administrative  support,  it  is  highly  likely  that  most  of  the  instructor’s  time  would  be 
spent  doing  trouble-shooting  and  problem  solving  with  respect  to  technology.  An  example  of  the  type  of 
things  the  graduate  assistants  address  include  whether  chat  programs  should  be  used  for  team  collaboration, 
which  chat  programs  work  best,  and  dealing  with  the  times  that  university  servers  go  down  and  students 
can’t  access  forums  or  the  class  web-site. 

Tuition  and  fees 

There  is  an  outstanding  question  as  to  whether  the  return  on  investment  outweighs  the  costs  of 
creating  and  implementing  web-based  courses.  The  need  to  generate  revenue  from  a Distance  Master’s 
program,  at  some  point,  will  become  a factor  that  impacts  the  design  and  adaptation  of  residential  courses 
to  the  web  environment.  The  fact  that  our  department  is  committed  to  this  program  in  order  to  provide 
instructional  design,  development  and  research  opportunities  for  our  students  means  that  this  Distance 
Master’s  course  and  the  program  as  a whole  have  an  entirely  different  impact  on  our  department  than 
would  be  the  case  if  the  primary  concern  of  our  department  was  to  generate  revenue  from  this  program. 

Recommendations 

Start  development  early 

The  initial  development  of  the  course  described  in  this  paper  occurred  over  a six-week  period  from 
July  to  August  2000,  and  required  approximately  500  hours  of  labor  from  a development  team  of  five 
individuals  and  the  instructor  of  the  course.  Obviously,  the  more  people  you  have  available  and  the  more 
time  you  have,  the  better.  Although  the  course  development  took  only  six  weeks,  the  administrative  and 
department-level  foundation  had  been  laid  over  the  previous  6-12  months.  Even  if  you  do  not  begin 
designing  right  away,  you  need  to  start  talking  to  the  stakeholders,  setting  out  requirements,  getting  faculty, 
staff  and  administrative  buy-in,  and  dealing  with  bigger-picture  issues. 

Confirm  capacity  of  technology  to  address  needs 

Confirm  capacity  of  technology  to  address  needs.  Does  the  university  provide  the  technological 
tools,  software  programs,  and  administrative  support  that  will  be  needed  to  create  a successful  distance 
learning  experience?  Make  sure  that  the  programs  and  software  you  choose  are  appropriate  for  the  delivery 
mechanisms,  for  your  students,  and  for  your  content.  While  not  asking  for  new  equipment  and  software 
costing  thousands  of  dollars,  require  a solid  minimum  set  of  technology  standards  that  your  students  will 
meet.  Provide  faculty  with  the  best  equipment  and  connection  possible,  from  the  office  and  from  home. 
Develop  policies  for  ownership  of  materials 

There  is  much  discussion  of  who  owns  what  in  the  creation  of  online  courses.  Each  university  has 
its  own  policy.  Make  sure  you  are  familiar  with  yours  and  that  your  faculty  agree  to  abide  by  its  terms. 
These  polices  can  act  as  disincentives  for  faculty  to  create  excellent  online  instructional  materials,  so 
communicate  with  administration  about  potentially  updating  and  modifying  policies  that  are  too  restrictive 
or  that  flout  general  intellectual  property  rules. 

Have  minimum  technology  standards  for  students 

As  was  previously  mentioned,  learning  at  a distance  requires  certain  tools  that  are  different  from 
those  a regular  student  may  need.  Stating  explicitly  what  students  will  require  is  useful  in  that  students  can 
compare  their  present  capabilities  to  the  minimum  standards  and  have  good  guidelines  on  upgrading. 
Additionally,  if  they  want  to  buy  a new  computer,  they  have  the  standards  at  hand.  Our  university,  like 
many  others,  has  special  hardware  and  software  deals  with  a variety  of  companies  that  the  Distance 
students  can  take  advantage  of.  In  our  team-based  program,  if  one  student  cannot  connect  or  complete  the 
work  because  he  doesn’t  have  the  software,  it  is  not  just  his  problem,  it  is  his  team’s  problem,  and  therefore 
the  class’s  problem. 

Provide  detailed  technology  training  for  learners 

Residential  students  and  faculty  have  enough  problems  with  getting  their  technology  to  work 
smoothly.  Students  learning  at  a distance  are  exponentially  challenged  to  learn  and  troubleshoot  their 
technology  problems.  In  the  residential  1ST  program,  students  with  questions  can  easily  find  someone  in 
the  lab  or  in  the  hallway  to  help  them.  The  Distance  learners  may  have  no  one  within  500  miles  who  is 
working  with  this  software  and  therefore  must  rely  only  on  himself  and  the  available  resources.  To  this  end, 
we  not  only  gave  students  a crash  course  in  much  of  the  software  and  tools  they  would  be  using,  but  also 
taught  them  how  to  trouble  shoot  and  where  to  go  for  help.  There  is  a telephone  help  desk  that  they  have 
access  to,  as  well  as  Indiana  University’s  Knowledge  Base,  an  award-winning  technology  information 
database. 


Provide  learners  with  guidelines  for  when  to  use  each  communication  vehicle 

We  offered  very  little  guidance  about  which  tools  were  best  suited,  according  to  our  research,  for 
what  kinds  of  communication.  Some  of  ourteams  did  everything  from  team  meetings  to  actual  writing  and 
development,  completely  synchronously,  which  ended  up  being  long  chat  sessions  that  were  useless  to 
them  later*  and  $100  conference  call  phone  bills.  This  is  not  surprising  that  students  will  take  a while  to 
figure  out  the  best  methods  for  communicating  in  the  new  way.  Because  students  are  used  to  meeting  face- 
to-face  and  talking,  they  assume  that  the  chat  format  works  for  every  kind  of  interaction.  In  the  future,  we 
will  offer  students  guidelines  of  when  to  use  the  Forum,  when  to  use  chat,  when  to  use  e-mail,  etc. 
Test-adapt-test-adapt-test-adapt 

It  is  highly  unlikely  that  the  first  version  of  the  course  website  created  will  be  the  best  or  most 
effective  one,  so  you  should  plan  for  and  engage  in  at  least  several  iterations  of  usability  testing  with 
representatives  of  the  target  audience,  and  use  the  results  of  those  tests  to  adapt  the  website  in  order  to 
improve  the  educational  experience  for  the  learners. 

Recognize  that  all  faculty  are  impacted  by  the  implementation  of  a single  course 

One  faculty  member  in  isolation  will  not  be  able  to  design,  develop  or  implement  a web-based 
course.  It  takes  the  support  of  an  entire  faculty  to  complete  this  process.  Additionally,  the  department  as  a 
whole  needs  to  have  buy-in  to  the  process  and  the  product.  If  there  are  faculty  members  who  see  the  online 
version  of  the  program  as  being  watered-down  and  therefore  less  valid  than  a “regular”  degree,  these 
concerns  need  to  be  addressed.  Chances  are  good  that  during  the  development  and  first  implementation  of 
online  courses,  the  lead  faculty  member  will  have  to  lessen  her  other  departmental  responsibilities,  and  the 
other  faculty  members  have  to  be  willing  and  able  to  pick  up  that  slack.  Although  some  faculty  may  never 
teach  in  the  online  format,  they  may  be  asked  to  “guest  lecture”  or  find  other  ways  to  interact  with  the 
Distance  students.  Even  if  they  have  absolutely  no  contact  wit  the  online  program,  their  lives  and  jobs  will 
be  affected  by  the  program  and  therefore  they  are  important  stakeholders  in  the  process. 

Conclusion 

The  process  of  adapting  a residential  course  in  Instructional  Systems  Technology  to  a web-based 
course  for  a distance  masters  program  has  been  a challenging,  and  at  times,  a difficult  experience.  This 
process  of  adaptation  has  required  that  the  instructor  and  the  instructional  design  team  grapple  with  issues 
of  technology  support,  software  capabilities,  the  collection  and  dissemination  of  course  resources,  the 
technological  skills  of  learners  and  faculty,  and  the  motivation  of  learners  and  faculty  to  create  and  sustain 
community.  It  has  required  that  the  instructor  move  out  of  the  comfort  zone  of  familiarity  with  the 
residential  learning  environment  to  consider  what  aspects  of  that  environment  are  most  appropriate  and  can 
best  be  replicated  in  an  on-line  environment,  and  what  unique  features  of  the  on-line  environment  can 
facilitate  learning. 

Each  adaptation  of  a residential  course  to  a web-based  environment  will  be  in  many  ways  unique 
and  non-generalizable,  as  is  the  case  of  adaptation  forthe  R522  course  which  has  been  described  here.  Yet, 
consideration  of  the  experiences  of  this  instructor  and  instructional  design  team  may  offer  some  insightto 
those  who  are  in  the  process  of  adapting  other  courses  to  web  environments  of  issues  that  impact 
adaptation,  including  course  objectives,  technological  capacity,  and  learner  skills  and  knowledge  related  to 
the  use  of  sophisticated  technology. 
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Human-centered  design  bill  of  rights  for  educators 


William  A.  Sugar,  Ph.  D. 
East  Carolina  University 


It  appears  to  be  a ceaseless  question  on  determining  the  best  ways  to  incorporate  technology 
within  public  schools.  Though  the  past  relationship  between  the  Instructional  Technology  discipline  and 
public  schools  has  been  tenuous  and  limited  (Heinich,  1995),  Willis,  Thompson,  and  Sadera  (1999) 
recently  present  encouraging  news.  They  summarize  the  current  research  of  a new  sub-discipline  within 
instructional  technology:  information  technology  and  teacher  education.  This  new  field  offers  effective 
solutions  to  encourage  technology  adoption  and  integration  within  public  schools.  This  article  presents 
another  potential  solution  by  proposing  a human-centered  technology  bill  of  rights  for  educators.  The 
intention  of  this  bill  of  rights  is  to  influence  educators*  beliefs  towards  technology  and  to  enable  educators 
to  confront  the  seemingly  mesmerizing  host  of  new  technologies  with  confidence. 

Instructional  technology  in  the  schools:  Allegiance  and  resistance 

Throughout  the  last  century,  pubic  support  for  the  use  of  new  technologies  in  the  classroom  has 
been  sympathetic  and  promising.  New  technological  gadgets  (e.g.,  Edison’s  kinetographs,  instructional 
television,  filmstrips,  videocassettes,  and  now,  DVD  players,  etc.)  have  been  designated  as  “champions” 
that  will  help  facilitate  the  ultimate  technology  integration  and  revolution  in  schools.  The  general  public 
tend  to  view  technology  as  being  “positive”  overall  (Kerr,  1 996).  This  allegiance  appears  to  be  somewhat 
of  a blind  frenzy  where  there  is  an  “initial  wave  of  enthusiasm  for  new  technology”  (Cuban,  1986,p.4).  In 
fact,  Segal  (1996)  describes  this  phenomenon  as  “technological  utopianism”  whereas  Mellon  (1999) 
characterizes  the  current  allegiance  to  technology  as  a worshiping  act. 

However,  the  “Field  of  Dreams”  syndrome4  does  not  apply  to  educational  settings.  Even  though 
new  technologies  are  proclaimed,  introduced,  purchased  and  adopted  by  schools,  teachers  are  resistant  to 
“modem”  technologies  and  are  reluctant  to  accept  new  technologies  (Cuban,  1 986,  p.  2).  In  most  cases, 
technology  is  “hurled”  at  teachers  when  “non-teachers”  introduce  and  originate  new  technologies  into 
schools  through  top-down  mandates  (Cuban,  1986,  p.  54).  This  is  currently  the  case.  Public  schools  are 
well  equipped  with  computers,  but  all  to  often  teachers  do  not  regularly  use  this  technology  in  their 
classrooms  (Cuban,  1993;  Office  of  Technology  Assessment,  1995).  Though  Becker  (1998)  reports 
promising  news  for  Internet  use  among  teachers,  universal  acceptance  of  new  technologies  among  teachers 
should  not  be  taken  for  granted. 

Immature  and  mature  views  of  technology 

Despite  this  initial  enthusiasm,  reluctance,  and  resistance,  individual  teachers  have  successfully 
adopted  and  integrated  technologies  within  their  classrooms.  However,  there  have  been  inconsistent 
technology  adoption  styles  and  usage  among  teachers  (Dexter,  Anderson,  and  Becker,  1 999,  p.  22 1 ; Office 
of  Technology  Assessment,  1995).  These  different  styles  can  be  categorized  as  an  immature  view  of 
technology  as  opposed  to  a mature  view.  An  immature  view  perceives  new  technology  as  an  add-on  or 
supplemental  activity  and  develops  routine  instructional  tasks  (Willis  & Mehlinger,  1 996,  p.  984).  With  a 
top-down  approach,  Fabry  and  Higgs  (1 997,  p.  389)  report  that  new  technologies  have  been  forced  into 
schools  which  “typically  results  in  superficial  adoption  rather  than  incorporating  the  substance.”  Ely 
( 1 997)  comments  that  new  technologies  (e.g.,  computers)  “rarely  supplant  other  media  and  methods”  (p. 
1 04).  A mature  viewpoint  of  technology  goes  beyond  this  superficiality  and  views  a particular  technology 
as  a tool  to  facilitate  learning  (Office  of  Technology  Assessment,  1 995).  Thus,  this  new  technology  is  not 
supplementing  a particular  curriculum,  but  is  in  the  process  of  revolutionizing  that  curriculum.  In  fact, 
Ertmer  (1999,  p.  47)  states  “in  general,  the  more  integrated  one’s  technology  use  becomes  the  more 
fundamental  the  required  changes.” 


4 “Field  of  Dreams”  refers  to  the  film  with  Kevin  Costner.  In  this  film,  Costner’s  character  builds  a baseball 
park  in  a com  field  with  the  expectation  that  people  will  come  once  the  field  is  built. 


Still,  whether  a school  adopts  an  immature  or  mature  perspective  of  technology,  the  critical 
variable  in  the  adoption  and  subsequent  integration  of  a particular  technology  is  the  teacher  (Fabry  & 
Higgs,  1 997;  Saye,  1998).  Teachers  must  be  convinced  of  the  feasibility  ofusing  a particular  technology 
before  this  adoption  and  integration  occurs  (Office  of  Technology  Assessment,  1995,  p.  71).  Teachers 
must  perceive  any  possible  change  (technological  or  non-technological)  as  being  helpful  to  their  current 
duties  (Saye,  1 998,  p.  2 1 1 ).  Ertmer,  Addison,  Lane,  Ross,  & Woods  ( 1 999,  p.  55)  concur  by  stating,  “if  the 
computers  do  not  teach  what  the  teacher  stresses,  teaches  things  the  teacher  does  not,  or  requires  types  of 
intelligent  activity  the  teacher  does  not  emphasize,  it  is  unlikely  the  teacher  will  assign  high  value  to  its 
use.”  Quite  possibly,  teacher’s  resistance  and  an  immature  perspective  towards  technology  could  be  solely 
dependent  on  teacher’s  acceptance.  Top-down  mandates  will  not  bring  about  a fully  technology-rich 
curriculum,  but  only  teacher’s  individual  decision  and  acceptance  of  a new  technology  will  be  the  guiding 
force. 

If  it  is  true  what  Willis  and  Mehlinger(l  996,  p.  978)  state  as  an  “universal  conclusion  that  teacher 
education,  particularly  pre-service  is  not  preparing  educators  to  work  in  a technology-enriched  classroom,” 
then  we  need  to  create  the  environment  in  which  teachers  can  make  rational  decisions  and  accept  a 
particular  technology.  In  fact,  Willis  (1993)  notes  that  we  have  moved  into  a new  era  of  educational 
computing  and  a new  way  of  asking  questions  about  technology  adoption  (p.  14).  We  must  now  start 
asking  questions  about  the  new  roles  and  responsibilities  of  a teacher  and  provide  training  to  support 
teacher  technology  adoption  and  integration  (Willis,  1993,  p.  28).  Fabry  and  Higgs  (1997)  note  that  in 
order  to  adopt  a mature  view  of  technology  teachers  “must  also  fundamentally  change  how  they  teach.”  (p. 
388). 

I take  an  optimistic  stance  and  believe  that  we  can  provide  an  environment  in  which  teachers  can 
individually  accept  and  maturely  integrate  technologies  into  their  curriculums.  In  order  to  facilitate  this 
interaction,  I propose  a technology  bill  of  rights  that  is  aimed  at  influencing  teachers’  belief  system.  Before 
this  bill  of  rights  and  its  principles  are  explicated,  a summary  of  factors  that  influence  and  impede 
technology  adoption,  as  well  as  a distinction  between  Ertmer’s  (1  999)  first-  and  second- orderbarriers must 
occur. 

Incentives  and  obstacles  for  integrating  technology 

Fortunately,  past  research  has  detailed  numerous  incentives  and  obstacles  towards  adopting  and 
integrating  technology  within  schools.  Table  1 summarizes  these  factors.  Some  obvious  influences  include 
tangible  factors,  such  as  additional  resources,  financial  support,  staff  development,  etc.  Teachers  need  to 
be  convinced  that  a particular  technology  will  solve  theirproblems  through  practical  means  (Cuban,  1986, 
p.  66).  In  addition  to  these  alterations,  tacit  improvements  need  to  be  implemented  to  promote  the  adoption 
of  new  technologies.  Some  of  these  proposed  improvements  include,  promoting  teacher  empowerment 
(Topp,  Mortenson,  & Grandgenett,  1 995,  p.  11);  providing  a comfortable  atmosphere  and  individualized 
attention  (Schrum  & Fitzgerald,  1 996);  creating  a comfort  zone  (Norum,  1 997)  and  other  similar  factors. 
Though  it  may  be  difficult  to  develop  a “comfort  zone”  for  teachers  using  technology,  these  tacit  changes 
may  be  more  significant  than  providing  additional  resources  or  offering  staff  development  technology 
courses.  Fullan  and  Stiegelbauer  ( 1 99 1 , p.  3 1 5)  note  that  "nothing  has  promised  so  much  and  has  been  so 
frustratingly  wasteful  as  the  thousands  of  workshops  and  conferences  that  led  to  no  significant  change  in 
practice  when  the  teachers  return  to  their  classrooms.”  Fundamental  technological  changes  could  be 
directly  linked  to  these  tacit  stimuli. 


Table  1 

Incentives  and  obstacles  affecting  technology  adoption  and  integration 


Incentives 


Obstacles 


• Adequate  equipment  and  resources  (Becker,  1994;  Fabry  & 

Higgs,  1997;  Hadley  & Sheingold,  1993;  Office  Of 
Technology  Assessment,  1995;  Topp,  Mortenson,  & 
Grandgenett,  1995) 

• Supportive  environment  that  allows  “risk  taking"  (Becker, 

1994;  Topp,  Mortenson,  & Grandgenett,  1995,  p.  12; 
Willis,  1993) 

• “Expectations  and  encouragement  are  vital  to  the  infusion  of 

technology  into  the  educational  process”  (Topp, 
Mortenson,  & Grandgenett , 1995,  p.  13) 

• Collegiality  among  users  (e.g.,  teachers)  (Becker,  1994) 

• Smaller  class  sizes  (Becker,  1994) 

• “Exemplary  teachers  were  in  schools  that  had  nearly  twice  as 

many  computer-  using  teachers  (Becker,  1994,  p.  303) 

• Personal  interest  (Becker,  1994;  Ertmer,  et.  al.  (1999) 

• Extra  time  (Fabry  & Higgs,  1997;  Hadley  & Sheingold, 

1 993;  Office  Of  Technology  Assessment,  1 995;  Schrum  & 
Fitzgerald,  1996;  Willis,  1993) 

• Staff  development  (Becker,  1994;  Fabry  & Higgs,  1997; 

Office  Of  Technology  Assessment,  1995;  Willis,  1993) 

• “Exemplary  teachers  simply  had  higher  standards  and  greater 

perceived  needs  than  did  the  other  computer  users.” 
(Becker,  1 994,  p.  3 1 5) 

• Attempt  to  reach  students  with  learning  or  attention  problems 

(Ertmer,  et.  al.,  1999) 

• Motivated  to  make  lessons  more  interesting  (Ertmer,  et.  al., 

1999) 

• Preparing  students  for  the  future.  (Ertmer,  et.  al.,  1999) 

• Staff  support  (Becker,  1994;  Office  Of  Technology 
Assessment,  1995) 

• Teachers  must  be  empowered  to  make  decisions  about 

technology  (Fabry  & Higgs,  1 997,  p.  390) 

• “Ownership  is  critical  to  success.”  (Willis,  1993,  p.  29) 

• “Follow  up  support  and  coaching  is  as  essential  to  effective 

staff  development  as  is  the  initial  learning  experience.” 
(Office  Of  Technology  Assessment,  1995,  P.  30) 


• Lack  of  technology  skills  and  knowledge  (Martinez  & 

Woods,  1995) 

• Lack  of  equipment  (Ertmer,  et.  al.,  1999) 

• Mismatch  with  classroom  style  (Ertmer,  et.  al.,  1999) 

• Lack  of  staff  development  (Ertmer,  et  al.,  1999;  Fabry  & 

Higgs,  1997;  Office  Of  Technology  Assessment,  1995; 
Topp,  Mortenson,  & Grandgenett,  1995) 

• Absence  of  incentive  or  improper  incentives  (Martinez  & 

Woods,  1995) 

• Absence  of  environmental  support  (Martinez  & Woods, 

1995) 

• Lack  of  motivation  (Martinez  & Woods,  1995) 

• Lack  of  equipment  (Ertmer,  et  al.,  1999) 

• Lack  of  time  (Ertmer,  et  al.,  1999;  Office  Of  Technology 
Assessment,  1995) 

• Lack  of  relevance  (Ertmer,  et  al.,  1999) 

• Lack  of  confidence  (Ertmer,  et  al.,  1999) 

• Lack  of  funding  (Fabry  & Higgs,  1997;  Office  Of 
Technology  Assessment,  1995) 

• Lack  of  access  (Fabry  & Higgs,  1 997;  Office  Of  Technology 

Assessment,  1995) 

• “Innate  dislike  for  change  (especially  change  mandated  from 

above)  is  the  most  basic  and  significant  barrier  to 
technology  integration”  (Fabry  & Higgs,  1997,  p.  388) 

• “Top  down  projects  tend  to  fail  over  time.”  (Willis,  1 993,  p. 

29) 

• Current  assessment  practices  (Office  Of  Technology 
Assessment,  1995) 


Peggy  Ertmer  (Ertmer,  1999;  Ertmer,  et.  al.  1999)  distinguishes  between  these  two  factors  as 
being  first-order  barriers  and  second-order  barriers  to  technology.  First-order  barriers  are  external  to  the 
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particular  environment,  such  as  securing  additional  resources  to  learn  more  about  a new  technology. 
Second-order  barriers  are  internal  to  the  school  setting  and  are  related  to  one’s  beliefs  about  the  use  of 
technology.  Second-order  barriers  “confront  fundamental  beliefs  about  current  practice,  thus  leading  to  new 
goais,  structures  or  roles”  (Ertmer,  1 999,  p.  48).  Erimer  notes  that  both  of  these  barriers  are  prevalent  in 
the  challenges  of  technology  adoption  and  both  types  of  barriers  need  to  be  confronted  simultaneously 
during  the  adoption  process  (Ertmer,  1999;  Ertmer,  et.  al.,  1999,  p.  70).  There  is  a distinct  interplay 
between  these  first-order  and  second-orderbarriers.  Ertmer,  et  al.  (1 999,  p.  55)  note  that  “researchers  have 
suggested  that  teachers’  beliefs  about  the  role  of  technology  in  the  classroom  may  eitherreduce  or  magnify 
the  effects  of  first-order  barriers”  and  that  “second-orderbarriers  may  persist  even  when  first-order  barriers 
are  removed”  (p.  70).  These  intrinsic  barriers  towards  technology  are  indefatigable  since  teachers’ 
fundamental  beliefs  towards  technology  are  more  difficult  to  modify.  However,  the  proper  removal  of 
these  second-order  barriers  can  be  a critical  variable  in  effective  technology  adoption.  This  elusive 
second-order  barrier  challenges  the  assumption  that  if  a school  provides  enough  tangible  resources  (e.g., 
money,  computers,  software,  etc.),  then  teachers  will  be  willing  to  use  new  technologies.  This  is  not  the 
case.  Teachers’  personal  beliefs  about  using  technology  will  greatly  influence  the  use  of  a particular 
technology.  Ertmer  (1999,  p.  51)  notes  that  “reduction  or  elimination  of  first-order  barriers  allowed 
second-order  barriers  or  issues  to  surface.”  The  Office  of  Technology  Assessment  also  notes  that  “many 
technology  rich  sites  continue  to  struggle  with  how  to  integrate  technology  into  the  curriculum”  (Office  Of 
T echnology  Assessment,  1 995,  p.  30).  Concentrating  on  shaping  teachers’  beliefs  about  using  technology 
and  removing  these  second-order  beliefs  is  a critical  variable  during  technology  adoption  and  integration 
process. 


Beliefs  about  technology  and  its  impact 

In  fact,  attitudes  and  beliefs  in  teaching  are  essential  in  understanding  how  a teacher  teaches, 
thinks,  and  learns  (Richardson,  1 996  p.  1 02).  Richardson  ( 1 996)  states  that  teachers’  beliefs  are  influenced 
by  personal  experience,  experience  with  schooling  and  instruction,  and  experience  with  formal  knowledge. 
Like  technology  barriers  and  influences  in  schools,  teachers’  beliefs  and  attitudes  towards  technology  also 
have  been  documented  in  past  studies.  Several  researchers  note  that  teachers  feel  uncomfortable  learning 
new  technologies  (e.g.,  Brush,  1 998,  p.  243;  Ertmer,  1 999,  p.  48;  Fabry  & Higgs,  1 997,  p.  389;  Schrum, 

1 999,  p.  85).  Willis  (1  993,  p.  28)  notes  that  “many  educators  feel  isolated  and  alone”  with  the  process  of 
learning  new  technologies.  Both  Carr-Chellman  and  Dyer  (2000)  and  Laffey  and  Musser  (1 998)  found  that 
pre-service  teachers  viewed  new  technologies  as  an  impediment  or  contradictory  to  effectively  teaching 
their  students.  Saye  ( 1 998)  offers  a typology  of  teachers’  beliefs  towards  technology.  Saye  developed  a 
continuum  of  two  types  of  beliefs:  accidental  tourists  and  voyageurs.  Accidental  tourists  seek  to  control 
and  adapt  technology  within  their  existing  teaching  structure  while  voyageurs  seek  to  use  technology  fora 
personal  challenge  and  willing  to  explore  and  experiment  with  technology  (Saye,  1 998,  p.  224).  Saye 
( 1 998)  found  that  most  teachers  were  accidental  tourists  and  tended  to  stay  within  their  own  teaching  style. 

Teachers’  attitudes  towards  technology  use  potentially  present  a formidable  obstacle.  These 
beliefs  impede  viewing  technology  as  a tool  and  promote  an  immature  view  about  technology.  That  is  to 
say,  if  a teacher  feels  apprehensive  about  using  technology  and  views  it  as  an  impediment  to  interacting 
with  students,  it  is  no  wonder  that  technology  is  being  used  exclusively  as  a supplement  and  within  the 
teacher’s  existing  teaching  style.  Alternatively,  successful  integration  of  technology  changes  educators’ 
view  of  the  teacher-student  relationship  and  alters  their  teaching  practices  (Dwyer,  1 996).  Becker  (1 998) 
found  that  teachers  who  purported  to  have  a constructivist  teaching  style,  tended  to  integrate  technology 
within  their  classroom.  Teachers’  willingness  to  change  is  a key  variable  in  successful  technology 
integration  (Marcinkiewicz,  1 994).  Though  both  Becker  (1994,  p.  291 ) and  Schrum  (1999,  p.  86)  agree 
that  with  experience,  teachers  may  eventually  become  comfortable  with  a particular  technology,  there  is  a 
missing,  critical  link  in  interacting  with  technology.  This  variable  is  teachers’  vision  about  how  technology 
should  be  utilized  in  their  classrooms. 

Cultivating  a positive  vision  towards  technology  can  be  a significant  factor  in  promoting  a mature 
view  of  technology  and  viewing  technology  as  a tool  (Ertmer,  1 999;  Office  Of  Technology  Assessment, 
1 995).  Ertmer  ( 1 999,  p.  54)  states  that  “one  of  the  important  steps  in  achieving  meaningful  technology  use 
is  the  development  of  a vision  of  how  to  use  technology  to  achieve  important  educational  goals.”  Ertmer 
(1999,  p.  54)  recommends  three  strategies  to  develop  a vision,  including  modeling,  reflection  and 


collaboration.  In  addition  to  these  strategies,  we  need  to  go  a step  furtherby  adopting  a set  ofbeliefsand 
fundamental  principles.  We  can  “jumpstart”  teachers’  positive  experiences  with  technology  by  proposing  a 
vision,  a set  of  beliefs,  and  fundamental  principles  on  the  use  of  technology  in  the  classroom.  I attempt  to 
encompass  these  values  by  proposing  a technology  bill  of  rights  of  educators.  Exercising  this  bill  of  rights, 
teachers  wili  become  comfortable  using  technology,  adopt  a mature  view  of  technology  and  start  using 
technology  as  a tool  in  their  classrooms. 

Technology  bill  of  rights  for  educators:  Human-centered  design  influences 

Summarized  in  Table  2,  this  proposed  technology  bill  of  rights  for  educators  is  directed  towards 
teachers’  use  of  technology  and  is  intended  to  be  a vehicle  to  change  their  existing  beliefs  (Ertmer’s 
second-order  barriers).  A brief  description  of  the  primary  influences  of  this  proposal  will  be  discussed 
before  each  of  these  rights  is  explicated.  Human-centered  design  principles  primarily  espoused  by  Donald 
Norman  (1988,  1993,  1 998)  is  the  main  philosophy  that  influences  this  bill  ofrights.  This  human-centered 
stance  proposes  an  attitudinal  shift  from  a reactive  stance  to  a more  proactive  stance  towards  using 
technology  in  the  classroom.  It  requires  designers  of  new  technologies  to  consider  the  needs  of  their  users 
as  a primary  factor  in  their  creation.  With  this  newly  adopted  human-centered  attitude,  teachers  will  more 
readily  integrate  "new"  technologies,  as  well  as  "old”  technologies  into  their  teaching  practices. 

The  term,  "human-centered”  is  synonymous  with  terms  such  as  ’’user-centered,”  "learner- 
centered.”  Essentially,  all  of  these  terms  reflect  the  belief  that  designers  must  create  products  based  upon 
that  teachers  can  use  and  are  upon  their  users’  (or  alternatively  upon  their  learners’  or  human’s)  perspective. 
Software  designers  in  particular  have  used  this  design  philosophy  and  methodology  as  early  as  the  1 970’s 
(Eason,  1988).  Donald  Norman  (1988,  1993,  1998)  who  originally  coined  the  term,  human-centered 
design , describes  this  methodology  as  a “process  of  product  development  that  starts  with  users  and  their 
needs  rather  than  with  technology.  The  goal  of  product  development  is  a technology  that  serves  the  user, 
where  the  technology  fits  the  task  and  the  complexity  is  that  of  the  task  not  of  the  tool”  (p.  1 85). 

This  philosophy  assumes  and  acknowledges  that  there  are  obstacles  to  effective  technology  use. 
To  remove  these  obstacles,  designers  can  improve  their  technological  products  based  upon  their  users’ 
perspectives.  To  accomplish  this,  designers  seek  actual  users  react  to  initial  prototypes  in  a lab  setting. 
Designers  then  interpret  these  reactions 

Table  2 

Proposed  human-centered  technology  bill  of  rights  for  educators 


• It  is  not  your  fault,  it  is  the  designers'  fault. 

• "Old"  technology  is  fine  to  use  as  long  it  is  effective. 

• Technology  will  conform  to  my  proposed  needs,  not  designer's  needs. 

• I am  a designer  of  technology,  rather  than  a user  of  technology. 

• Appropriate  technology  is  redundant  or  impractical  technology  is  an  oxymoron. 

• The  sole  purpose  of  technology  is  to  help  me  be  more  creative. 

• It  is  fine  to  make  ’’errors"  with  technology;  technology  will  adapt  to  my  mistakes. 

• Technology  is  designed  and  used  to  solve  my  problems. 

• The  more  active  technology  user  I am,  the  more  effective  the  technology  will  be. 


and  redesign  theirprototypes  to  better  accommodate  their  users’  needs.  Human-  centered  designers  are 
encouraged  to  "test  early  and  often"  (Nielsen,  1993). 

Similar  to  Tripp  and  Bichelmeyer’s  (1990)  rapid  prototyping  process,  designers  potentially  could  go 
through  several  iterations  in  gaining  information  about  theirusers’  perspectives.  Eventually,  designers  will 
create  an  effective  final  product. 

There  are  several  assumptions  about  designers  and  users  related  to  this  design  philosophy.  One  is 
that  designers  are  servingusers  and  their  needs.  Users  are  more  proactive  whereas  designers  need  to  be 
reactive  to  their  users’  requirements.  Technology  is  expected  to  work  effectively  for  its  users  and  be 
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intuitive.  If  a particular  technology  does  not  work  or  is  counter-intuitive,  then  it  needs  to  be  changed.  The 
users  do  not  need  to  be  changed.  Early  participation  from  actual  users  is  encouraged  and  promoted. 
Detected  errors  in  early  prototyping  sessions  are  positive  whereas  early  prototyping  sessions  that  do  not 
yield  problems  are  suspect. 

Proposed  human-centered  technology  bill  of  rights 

This  bill  of  rights  is  written  specifically  for  teachers  and  administrators  to  improve  their  attitude 
towards  the  use  of  new  technologies.  This  transformation  will  lead  these  educators  to  successfully 
integrate  these  technologies  in  their  curriculums  and  future  classrooms.  These  nine  rights  are  described 
below. 


It  is  not  your  fault , it  is  the  designers'  fault:  This  principle  addresses  the  “blame  syndrome”  in 
which  educators  and  otherusers  usually  blame  themselves  if  they  cannot  successfully  operate  a particular 
technology.  For  example,  students  in  my  online  Introduction  to  the  Internet  course  often  apologize  fornot 
performing  a particular  task,  such  as  properly  uploading  their  web  pages  to  a server.  They  tend  to  accept 
the  blame  fornot  being  successful  in  this  process.  From  a human-centered  design  perspective,  this  belief  is 
fallacious.  If  a user  cannot  navigate  properly  through  a particular  technology,  then  part  of  (if  not  all) 
responsibility  lies  with  the  designer.  Too  often,  educators  blame  themselves  for  doing  something  “wrong” 
with  technology.  They  might  have  pushed  the  “wrong”  button,  typed  in  the  “wrong”  command,  or  clicked 
the  “wrong”  icon.  This  error  points  to  the  fact  that  the  particular  designer  did  not  originally  anticipate  how 
we  would  use  it.  We  were  not  “wrong”,  but  our  mental  models  (Johnson-Laird,  1 983;  Norman,  1 986)  of 
this  process  did  not  correspond  to  the  designer’s  own  mental  model.  Since  we  did  not  originally  conceive 
ofthe  particular  technology,  incongruent  mental  models  are  not  our  responsibility.  It  is  true  that  educators 
need  to  learn  how  to  use  particular  technologies  (e.g.,  word  processors)  through  proper  training  (e.g., 
books,  workshops,  etc.)  and  designers  are  not  responsible  for  this  instruction.  However,  if  teachers  cannot 
utilize  a particular  technology  properly,  then  designers  must  accept  this  responsibility  and  redesign  the  next 
version  to  better  fit  their  target  audience’s  mental  model. 

“Old"  technology  is  fine  to  use,  as  long  it  is  effective.  Too  often,  an  erroneous  belief  is  that  the 
latest  version  of  a software  program  or  another  technology  is  universally  preferable  to  an  older  version  or 
an  older  technology.  But,  this  is  not  always  the  case.  Sometimes  an  older  technology  (e.g.,  a chalkboard) 
may  be  more  effective  than  its  more  recent  counterpart  (e.g.,  PowerPoint).  If  a teacher  can  teach  an 
effective  lesson  using  “old”  technology  (e.g.,  overhead  projectors,  chalkboard,  slide  projectors,  etc.)  then,  it 
is  acceptable  to  use.  Why?  Blindlyjumping  on  the  bandwagon  to  use  the  latest  technology,  especially  if  it 
is  costly  and  potentially  ineffective  makes  no  sense. 

This  is  especially  true  in  the  case  of  a high  school  in  the  southeast  region  of  the  United  States. 
Recently,  two  high  schools  merged  into  one  unified  high  school.  This  new  high  school  was  technologically 
upgraded  with  new  computers  and  related  equipment.  Teachers  in  both  schools  were  told  not  to  bring  any 
“old”  technology  to  this  new  school.  After  a few  weeks  of  this  school  merger,  some  teachers  were 
requesting  and  searching  for  their  “old”  technology.  Teachers  were  not  totally  resistant  to  this  new 
environment,  but  they  have  successfully  designed  effective  lessons  with  their“old”  technology.  The  key 
word  is  effective,  as  well  as  efficient.  As  long  as  “old”  technology  is  effective  and  efficient,  it  is  acceptable 
to  use.  However,  there  is  an  amendment  to  this  particular  right.  If  “old”  technology  is  not  effective  and/or 
efficient,  then  teachers  must  be  willing  to  get  rid  of  this  “old”  technology  and  replace  it  with  a “new” 
technology. 

Technology > will  conform  to  my  proposed  needs,  not  designer's  needs:  Notice  this  relationship. 
Who  is  conforming?  Technology  should  conform  to  our  needs  as  opposed  our  needs  conforming  to 
technology.  If  designers  of  a particular  technology  want  to  create  a successful  product,  they  must  attempt 
to  conform  to  our  needs.  This  right  is  directly  related  to  Donald  Norman's  human-centered  motto  for  the 
twentieth-first  century.  Norman  (1993)  notes  the  1933  Chicago  World’s  Fair  motto  was:  Science  Finds, 
Industry  Applies,  Man  Conforms.  Norman  proposes  a twentieth-first  century  human-centered  motto.  Itis: 
People  Propose,  Science  Studies,  Technology  Conforms.  This  new  motto  now  implies  that  we  must  be 
proactive  in  our  expectations  of  new  technology.  If  a technology  is  going  to  be  successful  and  effective  in 
schools,  it  now  must  conform  to  the  needs  of  teachers,  administrators,  and  students.  We  should  expect 
nothing  less  from  new  or  old  technologies. 


/ am  a designer  of  technology , rather  than  a user  of  technology:  This  right  also  represents  a shift 
in  perspective.  Teachers  are  naturally  “designers”  rather  than  “users,”  They  continually  design  new 
lessons,  curriculum  guides,  instructional  modules,  etc.  They  rarely  take  the  reactive  stance  of  users  in  their 
profession.  When  interacting  with  technology,  educators  should  adopt  the  proactive  “designer^  viewpoint 
as  opposed  to  reactive  “user”  viewpoint,  Wilson  (1999,  p.  1 6)  concurs  by  stating  that  “end  users  [teachers] 
need  to  think  like  designers  as  well  as  consumers,”  As  a “designer”,  teachers  should  expect  that  technology 
would  enable  them  to  be  successful  in  their  designer  roles  as  instructors.  Though  they  may  be  “using”  a 
particular  technology,  teachers  should  continually  ask  how  can  this  technology  enable  them  to  become  a 
more  effective  “designers.” 

Appropriate  technology  is  redundant  or  impractical  technology  is  an  oxymoron : This  right  is  a 
potentially  puzzling  statement,  but  actually  points  to  the  specific  nature  of  technology.  As  Ely  ( 1 997) 
notes,  too  often  technology  is  referred  to  as  an  object  (i.e.,  hardware,  software).  However,  if  one  looks  up 
the  definition  of  technology,  one  realizes  that  technology  is  a process  as  opposed  to  an  actual  “thing.”  This 
particular  right  describes  two  qualities  of  this  process.  One  aspect  of  technology  is  that  it  should  be  used 
for  appropriate  reasons.  We  must  adopt  the  perspective  that  it  is  preposterous  to  think  one  would  use 
technology  for  inappropriate  reasons.  True,  it  is  conceivable  for  someone  to  use  technology  for 
inappropriate  reasons  (e.g.,  using  a computer  to  add  single  digit  numbers),  but  why?  This  proposed  right 
implies  that  educators  will  utilize  technology  only  for  appropriate  reasons.  Otherwise,  it  is  a waste  of  time. 
The  second  aspect  of  technology  is  the  issue  of  practicality.  The  nature  of  technology  is  to  solve  practical 
problems.  In  fact,  the  nature  of  the  field  of  Instructional  Technology  is  to  solve  practical  problems  in 
instructional  settings,  as  opposed  to  solving  obtuse  problems  in  instructional  settings.  Again,  it  would  be  a 
pointless  activity  if  educators  used  technology  to  conceive  of  impractical  solutions. 

It  is  fine  to  make  ''errors”  with  technology ; technology  will  adapt  to  my  mistakes:  This  goes  back 
to  the  issue  of  technology  conforming  to  our  needs.  If  there  are  "mistakes”,  then  the  next  version  must 
accommodate  for  these  errors.  If  there  is  an  error  message  for  a particular  technology  or  if  something  goes 
"wrong",  then  the  responsibility  falls  directly  on  the  shoulders  of  the  designer  who  created  this  new 
technology.  One  of  my  colleagues  has  the  following  quote  in  her  office:  “technology  teaches  you 
patience.”  This  adage  embodies  the  frustration  that  most  of  us  probably  have  experienced  with  current 
technology.  I am  sure  that  I have  spent  countless  hours  with  a new  technology  (e.g.,  software  program) 
when  I could  have  done  the  same  task  in  half  of  the  time  using  another  means.  I was  not  making 
“mistakes”,  but  learning  to  become  more  patient  with  this  new  technology.  This  right  represents  a reversal 
of  roles.  Instead  of  making  an  “error”,  we  now  must  become  more  patient  with  the  designers’  ill-intended 
errors.  We  also  must  expect  that  designers  will  learn  their  mistakes  and  adapt  their  revised  versions 
accordingly. 

Technology  is  designed  to  solve  my  problems:  This  right  directly  equates  technology  as  being  a 
“tool..”  In  fact,  if  technology  is  not  being  used  as  a tool,  then  it  probably  is  being  misused.  Technology  is 
the  means  to  provide  practical  solutions.  The  purpose  of  this  “tool”  is  to  solve  problems.  By  adapting  this 
stance,  educators  can  make  clear  decisions  on  how  to  use  a particular  technology  for  a particular  situation. 
If  a technology  is  not  going  to  solve  a problem,  then  there  is  no  reason  to  use  it.  Administrators  should  not 
expect  all  of  their  teachers  to  use  a particular  technology  if  it  doesn’t  solve  their  problems.  The  emphasis  is 
on  selecting  the  right  tool  to  solve  a particularproblem.  This  right  will  give  a voice  to  those  educators  who 
do  not  feel  compelled  to  join  the  latest  technology  bandwagon,  but  give  them  credence  to  make  an 
informed  decision  in  selecting  the  appropriate  technology  to  solve  their  particular  problems. 

Technology  is  designed  to  help  me  be  more  creative:  In  addition  to  solving  problems,  educators 
should  expect  that  their  technology  should  facilitate  creativity  and  to  become  more  effective  instructors. 
Thus,  technology  should  be  designed  so  that  educators  can  be  creative  problem  solvers.  This  is  related  to 
Norman’ s ( 1 993)  “smart,  not  dumb”  concept,  Norman  notes  that  technology  could  either  make  us  “smart” 
or  “dumb”.  Technology  could  entrap  us  in  a senseless  stupor  such  as  television  viewers  staying 
mesmerized  in  one  spot  for  countless  hours.  Or  technology  could  enable  us  to  improve  and  illuminate  our 
selves  where  students  could  interpret  the  Declaration  of  Independence  from  multiple  perspectives.  It  is  no 
question  that  we  need  to  demand  the  latter  option.  If  a particular  technology  does  not  make  educators  and 
subsequently,  students  “smart”,  then,  there  is  no  reason  to  use  it. 


The  more  active  technology  user  1 am,  the  more  effective  the  technology  will  be  : This  is  another  quality 
of  effective  technology  and  related  to  the  “smart”  issue.  Educators  should  expect  to  be  active  partners  with 
a particular  technology.  One  should  not  be  “dormant”  users  and  interact  with  a technology  as  a "page 
turner".  Designers  should  create  a dynamic  environment  where  their  users  are  engaged.  In  fact,  “smart” 
users  occur  when  they  are  engaged  with  a particular  content  area  within  a dynamic  environment.  With  this 
expectation,  educators  not  only  will  be  active  technology  users,  but  their  students  also  will  be. 

Technology  bill  of  rights  for  educators:  Summary 

Each  of  these  nine  rights  is  deliberately  directed  towards  influencing  teachers’  attitude  toward 
technology  use.  If  you  consider  the  reasoning  of  each  right,  the  cognitive  rationale  might  be  lacking. 
However,  this  was  intended.  The  description  and  explanation  of  these  rights  resembled  more  of  a “pep 
talk”,  sermon,  or  another  similar  affective  oral  exposition.  The  intent  of  these  rights  are  to  influence 
teachers’  beliefs,  introduce  new  principles  on  how  to  view  technology,  empathize  with  teachers’ 
frustrations  with  technology,  and  inspire  teachers  to  consider  an  alternative  perspective  of  technology.  A 
more  comprehensive  cognitive  justification  was  consciously  withheld  in  order  to  focus  on  influencing 
educators’  affective  domain. 

With  the  assumption  that  effective  technology  adoption  involves  changing  both  attitudes  and 
behaviors  (Richardson,  1996),  this  proposed  technology  bill  of  rights  is  concentrated  on  influencing 
teacher's  attitudes  and  consequently,  changing  teachers’  behaviors  toward  technology.  By  changing  one’s 
attitude  towards  the  use  of  technology  within  schools,  teachers  could potentially  remove  several  obstacles 
towards  effective  technology  adoption  and  integration.  The  proposed  attitude  shift,  as  reflected  in  this  bill 
of  rights,  gives  teachers  the  opportunity  to  take  a fresh  stance  towards  use  the  new  technologies,  adopt  their 
own  vision  about  technology,  and  gives  them  the  opportunity  to  clearly  distinguish  whetherthey  should 
adopt  these  technologies  or  not. 

Future  directions  and  a possible  covenant 

Another  common  qual  ity  of  this  proposed  bill  of  rights  is  its  tentativeness.  This  bill  of  rights  is  in 
its  infancy  stage  and  needs  further  input.  Though  based  primarily  upon  human-centered  design  principles, 
no  empirical  studies  have  been  conducted  to  evaluate  the  efficacy  of  these  rights  in  changing  teachers’ 
attitudes  in  adopting  and  integrating  technologies.  There  are  obvious  next  steps  for  this  type  of  evaluation. 
More  research  on  teachers’  existing  attitudes  towards  technology  could  take  place.  A comparison  of  these 
existing  attitudes  and  proposed  attitudes  espoused  from  this  bill  of  rights  can  be  made.  From  this 
comparison,  a list  of  interventions  could  be  created  in  order  for  teachers  to  potentially  adopt  this  bill  of 
rights  in  their  own  teaching  practices.  Then,  the  next  question  would  be  to  distinguish  which  rights  enable 
teachers  to  become  effective  technology  adopters  and  integrators.  Do  any  of  these  rights  influence 
teachers’  behaviors  towards  technology? 

This  is  one  strategy.  Other  similar  strategies  and  input  on  this  bill  of  rights  are  welcomed  and 
encouraged.  The  main  point  is  to  further  the  discussion  on  encouraging  effective  technology  integration 
and  to  encourage  mature  perspectives  of  technology  amongst  educators.  Similar  to  the  infancy  of  our  own 
country,  our  national  bill  of  rights  was  debated,  discussed  and  altered.  This  latest  bill  of  rights  could  be  a 
catalyst  for  further  discussions  on  how  to  promote  effective  technology  adoption  and  integration  within  the 
public  schools. 

To  create  an  environment  of  effective  technology  adoption  and  integration,  we  must  focus  on 
eliminating  first-order  technology  barriers,  as  well  as  second-order  technology  barriers.  Donald  Norman’s 
human-centered  design  philosophy  could  be  a powerful  tool  in  influencing  educators’  perspective  towards 
technology.  This  human-centered  technology  bill  of  rights  is  a potential  means  to  change  the  way  teachers 
look  at  technology  and  its  use  in  education.  This  bill  of  rights  also  could  cause  teachers  to  adopt  a more 
mature  view  of  technology.  Adopting  the  principles  outlined  in  this  bill  of  rights  could  help  educators  to 
view  technology  as  a tool  as  opposed  to  other  immature  perceptions.  This  bill  of  rights  potentially  could  be 
a covenant  between  educators,  administrators,  and  designers  of  technology.  That  is,  a proclamation  on  how 
these  individuals  need  to  change  their  perspectives  towards  the  use  of  technology  in  schools.  In  his 
examination  of  the  adoption  of  twentieth  century  instructional  technologies,  Larry  Cuban  (1986,  1 993) 
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notes  that  very  little  technology  integration  has  occurred  in  schools,  since  the  fundamental  goals  and 
understandings  of  education  have  not  changed.  Possibly,  this  bill  of  rights  can  be  adopted  by  educators, 
administrators,  designers  and  alter  this  trend. 
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Background 

The  Web  CT  version  of  the  course  discussed  below  is  the  work  of  three  people  at  the  University  of 
South  Florida  (USF).  They  are  Barbara  Spector,  a professor  of science  education,  Barbara  Lewis, 
an  instructional  designer  and  computer  expert  in  the  Educational  Outreach  department,  and  Ruth 
Burkett  a doctoral  candidate  in  instructional  technology >.  They  came  together  18  months  ago  when 
Barbara  Lewis  arrived  at  USF.  Dr.  Spector  told  her  that  a need  had  developed  to  make  her 
science/technology/society  interaction  (STS)  course  available  to  students  through  distance 
learning  online.  Dr.  Spector  explicitly  stated  she  believed  it  could  not  be  done  without  totally 
ruining  the  course,  because  the  course  had  been  designed  on  constructivist  learning  theoty.  It  had 
been  taught  over  five  years  with  several  instructors  in  the  room  at  one  time  and  was  highly 
interactive.  Success  depended  on  developing  a community  of  co-learners  among  the  students  and 
the  instructors  and  establishing  an  environment  of  trust,  which  she  believed  could  not  be  done 
without  consistent  live  contact. 

Ms.  Lewis  said  if  Dr.  Spector  could  articulate  what  it  took  to  accomplish  what  was  done  in  a face- 
to-face  classroom,  she  could  make  it  happen  on  the  Web.  This  report  attests  to  the  fact  that  Ms. 
Lewis  was  right. 

Assumptions  about  constructivism 

The  following  are  assumptions  about  human  constructivism  (Novak,  1994),  the  theoretical  basis 
for  the  design  of  this  course.  Humans  leam  by  building  on  their  prior  knowledge.  They  are  not  empty 
vessels  into  which  we  pour  knowledge.  Their  prior  knowledge  is  organized  in  a cognitive  framework  that 
influences  what  a person  can  leam  and  the  ease  with  which  he/she  can  leam  it.  Each  person’s  cognitive 
framework  is  idiosyncratic  and  influences  (a)  the  items  to  which  a person  will  attend  in  his/her 
environment,  (b)  the  way  the  person  is  inclined  to  gather  and  process  data  about  those  items,  and  (c)  the 
Interpretation,  or  meaning,  one  makes  from  the  data  gathered.  Thus  the  quality  and  quantity  of  knowl  edge 
one  constructs  from  a learning  opportunity  is  greatly  influenced  by  how  much  of  a match  there  is  between 
one’s  cognitive  framework  and  the  structure  of  the  learning  opportunities  available. 

Designing  the  Web  site  for  constructivist  learning 

Building  on  the  above  understanding  of  constructivism,  the  Web  site  was  developed  for  the 
comfort  of  learners  regardless  of  the  level  of  a person’s  prior  knowledge  or  how  a person  was  inclined  to 
gather  and  process  data  (learning  approaches).  Learning  approaches  considered  in  the  design  included 
visual,  auditory,  or  kinesthetic,  sequential  or  global,  concrete  or  abstract,  sensing  or  intuitive,  and 
interactive  or  reflective. 

An  accommodation  for  expected  variations  in  prior  knowledge  related  to  STS  included  “just  on 
time  delivery”  of  supporting  information  to  shore  up  students’  idiosyncratic  conceptual  frameworks.  An 
accommodation  for  varied  approaches  to  learning  was  to  organize  resources  on  the  Web  site  such  that 
students  were  encouraged  to  make  choices  and  determine  their  own  learning  pathways  through  course 
materials,  unrestricted  in  time  or  sequence  by  the  instructor.  Additionally,  students  gathered  data  from  a 
variety  of  public  media  sources  of  their  choosing  each  week,  site  explorations  to  business  and  industry 
organizations  of  interest  to  them  in  the  community,  and  informal  education  agencies  and  local  schools  of 
their  choosing. 

Structuring  the  course  as  an  open-ended  inquiry  facilitated  students’  choices  of  learning  pathways. 
The  open-ended  question  for  this  inquiry  was:  “What  is  science,  technology,  and  society  interaction  and  its 
relationship  to  science  teaching?”  This  format  enabled  each  learner  to  begin  with  his/her  own  level  of  prior 
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knowledge,  use  whatever  learning  approach  was  most  comfortable,  work  with  others  in  the  course  in 
whatever  configuration  was  preferred,  and  pursue  items  of  interest.  In  these  ways  students  were  empowered 
to  take  charge  of  their  own  meaning  making,  the  goal  of  human  constructivism  (Novak,  1 994). 

The  combination  of  a person’s  idiosyncratic  cognitive  framework,  preferred  approach  to  a learning 
opportunity,  and  the  personal  pathways  selected  through  the  resources  led  to  individuals  making  sense  of 
data  in  a variety  of  ways.  Sharing  these  multiple  perspectives  within  the  group  enabled  each  learner  to 
make  more  connections  than  he/she  originally  generated,  thus  making  richer  meaning  and  deeper 
understanding.  Sharing  these  multiple  perspectives  and  how  they  were  derived  encouraged  members  of  the 
group  to  explore  other  learning  approaches  and  additional  pathways. 

Thus  the  course  is  iterative.  Learners  explore  resources,  gather  data,  make 
interpretations,  share  their  interpretations  through  Web  CT’s  forum,  receive  feedback 
from  this  learning  community,  explore  more  resources,  reinterpret  their  data,  and  share 
their  ideas  again.  The  process  is  repeated  each  week  as  learners’  make  more  connections 
to  deepen  the  understandings  they  are  constructing  and  generate  their  personal  grounded 
theory  of  STS.  While  progressing  through  learning  opportunities,  they  build  on  their  own 
prior  knowledge  and  experience  the  interactive  nature  of  learning  as  they  revise  and 
improve  the  quality  of  their  thinking  and  understanding  (Spector  & Burkett,  2000). 

Structure  of  the  Web  site 

Web  CT  was  used  as  the  course  shell  because  the  university  provided  the  software  and  workshops 
for  faculty  as  well  as  continuing  technical  support  for  faculty  and  students.  This  shell  provided  a password- 
protected  area  necessary  to  obtain  permission  for  posting  copyright-protected  materials.  The  course  used  all 
of  the  tools  available  in  the  software  package  except  the  quiz. 

Welcome  to  SCE  4237 

Sc/ence/T echnology/Society  Interaction 


R.A.T.S.  (Read  All  The  Screen):  Use  scroll  bars  to  view  entire  pages. 


© Barbara  S.  Spector,  Ph.  D,  University  of  South  Florida,  Tampa,  FL  33620 
Ruth  S.  Burkett,  Graduate  Assistant 
Support  by:  Barbara  A.  Lewis 

Figure  1 
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In  this  STS  course  the  students  developed  awareness  of  science  and  technology  as  human 
enterprises  that  take  place  in  social,  environmental,  and  historical  contexts.  The  course  had  four  main 
pages  (Figure  1)  with  more  than  270  linked  pages.  A student  entered  the  main  homepage  and  selected  the 
area  he/she  wished  to  explore. 

The  Syllabus  (Figure  2)  contained  general  information  about  the  course.  It  was  setup  with  an 
outline  on  the  left  for  the  students  to  click  on  information  to  view  in  any  order  or  to  use  the  navigational 
tools  to  scroll  through  the  syllabus  in  sequential  order.  The  syllabus  also  contained  active  links  to  enhance 
and  fill  gaps  in  prior  knowledge.  For  example,  if  a student  did  not  know  how  to  create  concept  maps  and 
one  required  learning  activity  asked  students  to  develop  successive  concept  maps  of  their  leaning  every 
three  weeks,  then  the  student  clicked  on  the  words  “concept  map”  to  view  instructions  and  criteria  for 
creating  a concept  map.  This  assignment,  along  with  other  open-ended  assignments,  was  a vehicle  for 
learning  as  well  as  a source  of  data  for  both  self-assessment  and  instructor-assessment. 


Syllabus 

♦ 1 . Syllabus 

2.  Come  obitcuves 

3.  Gtatnd  Vet 
Directions 

4.  Assumptions 

5.  Appendix  A: 
Comparison  of 
P>r*digms 

6.  Content  outline 

6.1.  Nature  & 
History  of 
STS 

6.2.  Examples 
of  STS 

6.3.  TeuMng 
STS 

7.  Organization  of 
Activities  . 

8.  Required  Learning 
Activities 

9.  Grading  Criteria 

10.  Reference  Material 


Science  / Technology  / Society  Interaction 
SCE4237(4  credits) 

College  Of  Education 

Syllabus  Fall  Semester,  2000 


This  is  a Web  Enhanced  Course:  For  computer  help  telephone  974-2996 


Instructor: 

Telephone: 


' Email: 


Dr.  Barbara  S.  Spector 

813*974-3533  and  leave  a message  with  the  secretary 
Including  you  phone  number  and  options  for  when  you  may  be 
reached. 

WebCT  email:  The  most  reliable  communication  is  via  email: 

In  an  emergency  situation  only,  email  me  at 
spector@tempest.coedu.usf.edu 


Required  Software: 


Course  Prerequisites: 


Inspiration  - available  for  purchase  at  the  USF  bookstore.  To 
start  immediately  you  can  download  the  preview  available  in 
Software  Helps  under  Student  Headquarters.  Caution:  Do  not 
wait  until  the  30  day  trial  is  up  to  purchase  the  program. 

(none) 


Class  meetings:  Class  meetings  on  the  Tampa  Campus  in  EDU  313  are  set  for 

the  following  dates  from  8:00  a.m.  to  12:00  p.m.:  August  29, 
2000  and  December  6, 2000. 

The  class  is  required  to  meet  at  Sugg  Middle  school  to 
participate  in  a two  day  science  class.  TBA. 


Figure  2 

The  Virtual  Resource  Center  (Figure  3)  was  the  core  of  the  course,  the  initial  source  of 
experiences  from  which  students  gathered  data  for  investigation  into  STS.  The  center  contained  print 
matter,  videotapes,  graphics,  interactive  media,  and  links  to  relevant  sites  on  the  World  Wide  Web.  These 
were  arranged  in  three  bins;  (a)  the  Nature  and  History  of  STS,  (b)  specific  Examples  of  STS,  and  (c) 
Teaching  STS.  The  division  into  separate  bins  (categories)  was  artificial  and  arbitrary  for  the  convenience 
of  study  (Spector  & Burkett,  2000).  Students  were  invited  to  explore  within  and  among  the  triangles,  in  any 
order,  to  construct  their  personal  theory  of  STS  grounded  in  data.  Questions  to  help  the  students  focus  their 
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inquiry  appeared  under  each  triangle.  The  icon  on  the  left  accessed  readings,  videos,  slide  presentations, 
Web  sites,  and  questions  focusing  on  the  nature  and  history  of  STS.  The  middle  icon  focused  on  specific 
examples  of  STS  relevant  to  humans  and  their  quality  of  life.  Examples  ranged  from  individuals,  to 
communities,  to  global  concerns.  The  icon  on  the  right  focused  on  teaching  science  at  middle  school,  high 
school,  and  higher  education  institutions. 


Virtual  Resource  Center 


This  is  your  initial  source  of  experiences  from  which  to  gather  data  for  your  investigation 
into  STS.  You  are  invited  to  explore  within  and  among  the  triangles  belov v in  any  order  to 
construct  your  personal  theory  of  STS  grounded  in  these  data.  It  may  be  advantageous  to 
sample  one  resource  from  each  triangle  early  in  your  investigation.  Questions  to  help  you 
focus  your  inquiry  appear  under  each  triangle  when  you  click  on  the  icons  below.  Selecting 
the  icon  on  the  left  v/ill  take  you  to  a triangle  with  readings , videos , slide  presentations,  web 
sites,  and  questions  focusing  on  the  nature  and  history  of  STS.  The  middle  icon  will  take 
you  to  a triangle  focusing  on  specific  examples  of  STS  relevant  to  humans  and  their  quality 
of  life.  Examples  range  from  individuals,  to  communities,  to  global  concerns:  The  icon  on 
the  right  will  take  you  to  a triangle  focusing  on  teaching  science  at  the  middle  school,  high 
school,  and  higher  education  institutions.  To  set  the  stage  for  your  journey  Into  STS  and  a 
plan  to  teach  it,  begin  by  clicking  on  the  * Teaching  STS'  icon,  the  What  & Why*  corner 
and  read  “Order  Out  of  Chaos:  Restructuring  Schooling  to  Reflect  Society's  Paradigm 
Shift. ' 


STS  STS  Teaching 

Nature  & History  Examples  STS 


Figure  3 


The  Student  Headquarters 
(Figure  4)  contained  (a)  links  to 
software  needed  to  access  course 
materials  (e.g.  Acrobat  for  print 
material  and  Media  Player  for  video) 
or  assist  in  completion  of  assignments 
(e.g.  Inspiration  for  concept  maps); 
(b)  ho\wto  directions  for  creating 
presentations  and  student  home 
pages;  and  (c)  other  tools  students  • 
found  useful  in  their  inquiry. 


SCE  4237:  Soisncm/ToohnotogySSoaiety  lntW9ation 


IMJcome  to  Student  Headquarters 


Seme  ottte  to.'  LfWa  recwvif  Adobe  Ai'totM  you  to  wW, 

trti&rtFVfbbi  *cro w C A* A * t 

4r  too*  tifson  to  opuntota 


finacu  PwwawQjMi 


Student  To" 


Software 


MroAKfroriLcttP* 


Figure  4 
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The  Communication  Center  (Figure  5)  facilitated  the  creation  of  a community  of  learners  where  there  was 
shared  responsibility  for  teaching  and  learning  with  a continuing  multi-directional  dialogue.  This  is  in 
contrast  to  the  traditional  one  -way  teacher  to  student  input  or  student  to  teacher  presentation.  The  bulletin 
board  forums  were  set  up  to  encourage  this  dialogue  through  interaction  with  other  students  and  the 
instructor.  Reflective  journals,  media  watches,  video  summaries,  and  other  learning  products  were  posted  in 
the  forums  as  stimuli  for  discussion.  Students  were  required  to  respond  to  each  other’s  postings  by 
critiquing  what  was  written  and  questioning  the  reasoning  and  evidence  presented.  The  instructor 
responded  similarly  as  a member  of  the  community,  not  as  the  authority.  Small  heterogeneous  study 
groups  and  groups  voluntarily  organized  to  do  tasks  were  free  to  use  the  chat  rooms  built  into  Web  CT  for 
synchronous  communication.  E-mail  was  used  on  occasions  when  personal,  one-on-one  communication 
was  needed. 


Figure  5 


Conclusion 

This  STS  course  demonstrated  a way  to  provide  constructivist  learning  and  teaching  online  to 
accommodate  varied  approaches  to  learning  and  differing  levels  of  prior  knowledge.  The  structure  of  this 
site  shifted  the  focus  from  teaching,  information  transmitted  by  the  teacher,  to  students  gathering 
information  in  an  on-time  delivery  mode  as  they  constructed  their  own  ideas  about  how 
science/technology/society  interact.  Key  features  that  enabled  this  constructivist  learning  we  re  structuring 
the  course  as  an  open-ended  inquiry,  creating  a virtual  resource  center  with  free  access,  embedding 
assessment  in  open-ended  developmental  instructional  tasks,  and  fostering  extensive  multi-directional 
interaction  among  all  members  of  the  learning  community  on  the  bulletin  board. 

Preliminary  findings  from  a study  in  progress  indicate  that  a community  of  co-learners  developed 
and  students  perceived  the  learning  environment  to  be  risk-free,  learned  to  trust  each  other  and  the 
instructor,  and  participated  in  intellectual  risk-taking  leading  to  significant  growth.  Thus  when  placing  a 
course  online,  it  is  feasible  to  maintain  the  integrity  of  a highly  interactive  course  developed  on 
constructivist  learning  theory. 


Communication  Center 


Online  Chat 
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The  problem  of  Transfer  of  Training  applied  to 
The  implementation  of  instructional  systems 

Pablo  Jose  Riboldi 
Brigham  Young  University 


Abstract 

Transfer  of  training  has  focused  on  learners  ' capability  to  apply  what  they  learned  to  other 
situations  outside  the  training  setting;  however,  the  implementation  process  can  also  be  thought  of 
as  a transfer  process.  When  teachers  mediate  the  instruction  as  they  implement  new  instruction, 
they  have  to  transfer  the  instruction  to  their  own  students.  This  presentation  reviews  several  useful 
concepts  about  transfer,  such  as  expected  transfer  and  elicited  transfer,  and  then  applies  them  to 
the  implementation  of  instruction.  A new  theoretical  construct,  transfer  alignment  coefficient,  is 
introduced  and  the  practical  implications  of  these  concepts  for  instructional  developers  is 
explored. 

Transfer  of  Training:  Problem  Definition 

Transfer  of  training  has  focused  on  learners’  capability  to  apply  what  they  learned  to  other 
situations  outside  the  training  setting;  however,  the  implementation  process  can  also  be  thought  of  as  a 
transfer  process.  When  teachers  mediate  the  instruction  as  they  implement  new  instruction,  they  have  to 
transfer  the  instruction  to  their  own  students.  This  presentation  reviews  several  useful  concepts  about 
transfer  and  then  applies  them  to  the  implementation  of  instruction.  A new  theoretical  construct,  transfer 
alignment,  is  introduced  and  the  practical  implications  of  these  concepts  for  instructional  developers  is 
explored. 

One  topic  from  the  literature  which  has  not  been  traditionally  associated  with  the  topic  of 
implementation  is  the  problem  of  transfer  of  training  (Bransford,  1979;  Clark  & Voogel,  1986;  Cronbach  & 
Snow,  1977;  Mayer,  1980).  Transfer  has  been  defined  as  learners’  capability  to  apply  what  they  learned  to 
other  situations  outside  the  training  setting.  The  two  kinds  of  transfer  are: 

1 . Near-transfer.  This  occurs  when  learners’  “performance  reaches  established  criterion  levels  on 
the  type  of  tasks  and  in  the  setting  defined  by  the  training”  (Clark  & Voogel,  1 986). 

2.  Far-transfer.  This  occurs  when  learners  are  “able  to  apply  [learned]  skills  in  contexts  that  are 
very  different  than  the  ones  encountered  during  training”  (Clark  & Voogel,  1986). 

Rumelhart  and  Norman  (1981)  imply  that  different  types  of  learning  objectives  may  contribute  to 
different  transfer  levels.  The  following  types  have  been  identified: 

1.  Procedural  objectives  are  those  that  represent  learning  where  sequences  of  steps  are  taught, 
procedures;  for  example,  how  to  diagnose  and  repair  an  engine.  Gagne  (1985)  calls  this  type  of  objective 
“intellectual  skills.”  Once  these  procedures  are  learned,  it  is  expected  they  can  be  applied  to  solve  similar 
situations,  such  as  diagnosing  and  repairing  another  similar  engine. 

2.  Declarative  objectives  are  those  concepts  and  principles  that  are  known,  e.g.,  the  principles  and 
concepts  of  internal  combustion  engines.  Gagne  (1985)  refers  to  declarative  knowledge  as  “verbal 
information.”  Declarative  tnowledge  is  expected  to  be  learned  so  that  it  can  be  generalized  to  solve 
unencountered  problems,  such  as  generalizing  the  principles  and  concepts  of  internal  combustion  engines 
to  the  science  of  gases. 

3.  Conditional  objectives  can  be  extrapolated  as  a third  type  of  learning  objective  not  mentioned 
by  Clark  and  Voogel  (1986)  but  identified  as  a kind  of  knowledge.  Conditional  objectives  allow  learning  of 
when  and  why  certain  procedural  or  declarative  knowledge  should  be  applied.  This  is  related  to  Gagne’s 
(1985)  cognitive  strategies.  Conditional  knowledge  is  at  the  crux  of  far-transfer  learning,  knowing  when 
and  why  to  apply  previously  learned  knowledge  in  solving  unencountered  problems. 

4.  Experiential  objectives  are  those  objectives  that  instead  of  specifying  a certain  learning 
outcome,  they  specify  a certain  type  of  experience  from  which  students  construct  meaning.  These  type  of 
objectives  can  be  associated  with  constructivist  instructional  strategies. 

In  the  opinion  of  Clark  and  Voogel  (1986), 
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Existing  instructional  design  models  (e.g.,  Reigeluth,  1983)  tend  to  focus  more 
effectively  on  near-transfer,  procedural  objectives,  than  on  far-transfer,  declarative  [and 
conditional]  ones.  The  reason  for  this  narrow  focus  of  instructional  design  models 
probably  lies  in  their  different  theoretical  origins.  Most  established  and  tested  design 
models  have  been  derived  from  behavioral  learning  theory.  Recent  models  tend  to  owe 
more  to  the  newer  but  more  tentative  cognitive  learning  theories.  Each  kind  of  theory 
proposes  a different  set  of  instructional  methods  to  facilitate  transfer,  (p.  1 1 5) 

Transfer  received  much  attention  from  the  behaviorist  point  of  view  in  the  first  half  of  the  20th 
century  (Ellis,  1965).  This  research  focused  on  the  identical  elements  model,  which  suggests  that  transfer 
improves  as  the  training  and  application  contexts  approach  in  their  similarities.  Many  behaviorally  based 
studies  provide  evidence  that  generally,  positive  transfer  increases  when  overall  training  and  application 
conditions  are  similar,  and  the  identical  elements-transfer  relationship  is  maintained  across  a variety  of 
learning  tasks  and  training-application  contexts. 

Criticisms  of  the  behavioral  transfer  model  (Bransford,  1979;  Cronbach  & Snow,  1977)  suggest 
that  only  near-transfer  is  facilitated  by  identical  elements  training  methods.  Cronbach  and  Snow  (1977) 
conducted  a review  of  behavioral  instructional  research.  They  concluded  that  students  with  a higher 
general-ability  were  able  to  achieve  far-transfer  in  these  studies,  but  only  as  a consequence  of  their  ability, 
and  not  as  a result  of  training.  Behavioral  training  methods,  it  seems,  primarily  tend  to  support  near-transfer 
learning  which  does  not  extend  to  different  content,  or  tasks  for  students  with  middle-  or  lower-abilities. 

Behavioral  methods  typically  include  a focus  on  behavioral  objectives  that  guide  instruction  and 
evaluation,  instruction  that  directs  and  monitors  student  progress,  and  shorter  instructional  segments  or 
frames.  These  instructional  segments  are  situations  where  language  is  simplified  and  standardized,  practice 
to  criterion  is  encouraged,  corrective  feedback  and  reinforcement  are  given,  and  tests  follow  instruction 
immediately.  In  most  cases,  behavioral  models  seem  to  encourage  the  teaching  of  procedures.  An  example 
of  this  may  be  found  in  statistics  courses  that  teach  solution  procedures  rather  than  underlying  principles 
and  concepts. 

Cognitive  instructional  models  seem  to  promote  far-  or  farther-transfer.  Cognitive  instructional 
methods  include  encouraging  discovery  strategies;  suggesting  the  use  of  previously  acquired  and 
decontextualized  skills  through,  for  example,  paraphrasing,  advance  organizers,  and  analogies,  and  an 
emphasis  on  testing  the  generalizability  of  learning.  DiVesta  and  Peverly  (1984),  in  one  of  the  few  studies 
to  compare  behavioral  and  cognitive  methods  for  promoting  transfer,  showed  that  far-transfer  was  highest 
in  the  cognitive  treatments  of  their  experiment.  Rumelhart  and  Norman  (1981)  have  described  a number  of 
studies  where  analogies  have  provided  efficient,  far-transfer  knowledge  of  new  information,  whereas 
behavioral  methods  seek  to  influence  learners’  overt  behavior  directly,  cognitive  methods  attempt  to 
promote  the  students’  mental  connection  of  new  knowledge  with  analogous,  useful,  and/or  meaningful 
prior  learning.  This  connecting  of  knowledge  from  dissimilar  contexts  may  be  the  reason  for  cognitive 
methods  promoting  far-transfer. 

Clark  and  Voogel  (1986)  conclude  that,  “Based  on  the  available  evidence,  it  is  reasonable  to 
assume  that  behavioral  methods  promote  near  transfer  and  procedural  learning;  whereas,  cognitive  methods 
yield  farther  transfer  and  declarative  [conditional]  learning.” 

Transfer  Alignment  Coefficient:  A New  Theoretical  Construct 

Transfer  research  has  focused  on  the  way  students  apply  learning  elsewhere;  however,  the 
implementation  process  can  also  be  thought  of  as  a transfer  process.  When  teachers  mediate  the  instruction 
as  they  implement  an  instructional  system,  they  transfer  the  information  to  their  own  students.  With  this 
perspective  in  mind,  two  related  concepts — expected  transfer  and  elicited  transfer — can  be  defined. 

Expected  transfer  is  the  type  of  transfer  (near  or  far)  necessary  to  effectively  transfer  different 
types  of  learning  outcomes  (procedural  or  declarative).  For  example,  the  expected  transfer  for  a declarative 
learning  outcome  is  far  transfer.  The  more  conceptual  the  learning  objective,  the  farther  the  expected 
transfer  would  be;  conversely,  the  more  procedural  the  learning  objective,  the  nearer  the  expected  transfer 
would  be.  Figure  6describes  this  concept. 

Elicited  transfer  is  the  type  of  transfer  (near  or  far)  most  likely  to  be  achieved  by  the  type  of 
instructional  strategy  (behavioral  or  cognitivist)  the  instructional  system  prescribes  teachers  to  use  for 
teaching  different  outcomes.  For  example,  if  the  instructional  system  uses  cognitive  strategies  for  the 
teacher  to  teach,  the  elicited  transfer  is  far.  The  more  cognitive  or  constructivist  the  instructional  strategy  of 
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the  system,  the  farther  the  elicited  transfer  would  be;  conversely,  the  more  behavioral  the  instructional 
strategy,  the  nearer  the  elicited  transfer  would  be.  Figure  7 portrays  this  concept. 

Therefore,  transfer  alignment,  I ’ is  the  correlation  coefficient  (-1  <7’<1)  between  the  expected  and 
the  elicited  transfer  per  learning  objective.  If  the  instructional  strategies  used  by  the  instructional  system  to 
teach  different  types  of  learning  objective  s elicit  the  same  type  of  transfer  as  would  be  expected  for  the  type 
of  objective,  then  the  transfer  alignment  would  be  strong  (close  to  1,  aligned).  If  there  is  no  relationship 
between  the  chosen  instructional  strategies  and  the  type  of  objectives  (for  example,  a uniform  behavioral 
approach  is  used  to  teach  all  sorts  of  learning  objectives),  transfer  alignment  would  be  null  (close  to  0, 
unaligned).  If  the  wrong  strategy  is  used  to  teach  the  objectives  or  to  teach  the  teachers  how  to  teach  these 
objectives  (as  in  the  case  when  the  instructional  system  behaviorally  prescribes  for  instructors  how  to  teach 
cognitivist  strategies  to  achieve  cognitivist  outcomes),  transfer  alignment  would  be  negatively  correlated 
(close  to  -1 , misaligned).  This  is  graphically  depicted  in  Figure  8 below. 

For  example,  if  the  instructional  system  deals  with  procedural  learning  outcomes  and  uses 
behavioral  teaching  methods,  then  teachers  only  need  to  achieve  near-transfer  of  the  instruction  to  their 
learners.  The  type  of  learning,  teaching  methodology,  and  transfer  required  are  aligned. 

On  the  other  hand,  if  the  instructional  system  deals  with  declarative  and  conditional  learning 
outcomes  but  uses  behavioral  teaching  methods,  or  the  system  behaviorally  requires  teachers  to  use 
cognitive  methods,  then  teachers  may  only  be  able  to  achieve  near-transfer  of  the  instruction  to  the  learners. 
Simultaneously,  they  must  realize  that  the  system  is  inadequate  in  achieving  the  level  of  far-transfer 
required  to  meet  the  declarative  and  conditional  learning  outcomes.  There  is  a misalignment  between  the 
type  of  learning,  teaching  methodology,  and  transfer  required. 

Transfer  Alignment,  Application  to  Implementation 

Why  would  transfer  alignment  be  important?  Achieving  the  right  type  of  transfer  is  an  integral 
part  of  the  learning  process.  If  learners  cannot  apply  what  they  learned  to  other  situations  outside  the 
training  setting,  then  we  cannot  claim  that  the  learning  objectives  have  been  achieved.  Clark  and  Voogel 
(1986)  showed  that  different  types  of  transfer,  near  or  far,  are  needed  to  learn  procedural  or  declarative 
objectives  and  that  different  instructional  strategies,  behavioral  or  cognitivist,  are  more  likely  to  promote 
different  types  of  transfer.  The  better  align  these  three  are,  the  more  likely  that  students  will  successfully 
transfer  their  knowledge.  Poor  transfer  alignment  is  like  hammering  nails  with  a screwdriver. 

For  teachers  charged  with  the  implementation  of  a new  instructional  system,  transfer  alignment 
refers  to  the  alignment  not  only  between  the  type  of  learning  objectives  and  the  teaching  strategy  they  are 
told  to  use  to  achieve  these  objectives,  but  also  between  the  instructional  objectives  and  the  instructional 
strategy  the  system  uses  to  tell  the  teacher  how  to  teach.  This  second  type  of  alignment  may  not  be  readily 
recognized  and  it  could  be  ignored  by  instructional  developers.  This  would  be  the  case,  for  example,  of  an 
instructional  system  which  tells  teachers  how  to  use  appropriate  cognitive  strategies  to  achieve  some 
declarative  learning  outcomes  but  it  does  so  by  prescribing  teachers’  behaviors.  Because  the  use  and 
application  of  cognitive  instructional  strategies  is  in  itself  a declarative  outcome,  unless  teachers  are  taught 
cognitively  how  to  use  these  strategies  they  will  be  unable  to  achieve  the  necessary  far  transfer  for  them  to 
apply  the  strategies  in  their  classrooms. 

What  effect  would  transfer  alignment  have  on  implementation?  A positive  alignment  would  have 
the  effect  of  increasing  implementation  because  teachers  would  be  more  likely  to  achieve  the  appropriate 
type  of  transfer  as  they  teach  with  the  instructional  system. 

This  presentation  explored  the  concepts  of  transfer  of  training  including  that  of  Transfer 
Alignment,  alignment  between  learning  outcomes,  instructional  strategies,  and  transfer,  as  applied  to  the 
implementation  process.  Several  implication  for  improving  the  development  of  instructional  systems  were 
offered. 
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Transfer  of  instructional  systems 


■ Expected  Transfer 

► The  type  of  transfer  (near  or  far)  necessary  to 
effectively  transfer  different  types  of  learning 
outcomes  (procedural  or  declarative). 


Nearer 


Procedural 


Expected  Transfer 

Declarative 

Type  of  Learning  Outcomes' 


► 

Farther 

Experiential 


Figure  6.  Expected  Transfer  as  a the  type  of  transfer  required  by  the  types  of  learning  outcomes. 
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Transfer  of  instructional  systems 


■ Elicited  Transfer 

► The  type  of  transfer  (near  or  far)  most  likely  to 
be  achieved  by  the  type  of  instructional 
strategy  (behavioral  or  cognitivist)  the 
instructional  system  prescribes  teachers  to 
use  for  teaching  different  outcomes. 


Nearer 


Behaviorist 


Elicited  Transfer 

Cognitivist 

Type  of  Instructional  Strategies 


► 

Farther 

Constructivist 


Figure  7.  Elicited  Transfer  as  the  type  of  transfer  promoted  by  the  system's  instructional  strategies. 


(near)  Expected  Transfer  (far) 

Procedural  Learning  Outcomes  Declarative 


Figure  8.  Transfer  alignment  as  the  correlation  coefficient  between  expected  and  elicited  transfer  per 
instructional  objective  for  three  different  instructional  systems. 
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Guidelines  for  Instructional  Sequencing  in  Emotional 
Literacy  Learning-Using  PATHS  Curriculum  as  an  Example 

Yann-Shya  Wu 

Indiana  University-Bloomington 


Abstract 

The  purpose  of  this  paper  is  to  provide  guidance  for  instructional  sequencing  in  emotional  literacy 
curricula.  First,  the  concepts  of  instructional  sequence  and  the  problems  involved  with 
instructional  sequence  in  affective  domain  of  learning  are  addressed . Then,  through  the  analysis 
of  the  emotional  literacy  curriculum  Promoting  Alternative  Thinking  Strategies  (PA  THS)  as  an 
exemplary  case,  general  principles  are  inferred  for  instructional  sequencing  in  emotional  literacy 
learning.  From  these,  principles  for  micro-level  (within-lesson)  sequencing  and  macro-level 
(curriculum)  sequencing  are  derived.  Factors  influencing  these  sequencing  principles  are 
discussed. 

1.  Introduction 

Within  the  wide  range  of  opinions  on  the  best  way  to  categorize  types  of  learning,  Bloom’s  (1 956) 
categorization  of  learning  into  three  domains— cognitive,  affective,  and  psychomotor-  is  the  most  widely 
accepted.  The  cognitive  domain  of  learning  “deals  with  the  recall  or  recognition  of  knowledge  and  the 
development  of  understandings  and  intellectual  abilities  and  skills”  (Reigeluth  & Moore,  1 999,  p.  52).  The 
affective  domain  of  learning  “refers  to  components  of  affective  development  focusing  on  internal  changes 
or  processes,  or  to  categories  of  behavior  within  affective  education  as  a process  or  end-product”  (Martin  & 
Reigeluth,  1 999,  p.  486).  The  psychomotor  domain  of  learning  “involves  athletic,  manual,  and  other  such 
physical  skills”  (Heinich,  Molenda,  & Russell,  1993,  p.  41).  Scholars,  researchers,  and  practitioners  in  the 
educational  fields  have  been  endeavoring  to  design  and  develop  curricula  and  instructional  methods  that 
will  promote  effective  and  efficient  learning.  Among  these  three  domains  of  learning,  cognitive  learning 
and  psychomotor  learning  have,  up  until  now,  received  the  most  scholarly  attention  and  benefited  from  the 
greatest  and  most  sophisticated  efforts  in  design,  development,  and  practice.  Meanwhile,  comparatively 
little  progress  has  been  made  toward  designing  and  developing  curricula  and  instructional  methods  in 
affective  learning  (Beane,  1990).  Among  the  major  reasons  for  this  state  of  affairs  are  the  currently 
uncertain  and  unclear  notions  of  the  proper  definition  and  scope  of  affective  learning,  and  a prevailing 
over-general  approach  that  makes  it  more  difficult  to  undertake  scholarly  research  in  this  area  (Bills,  1 976; 
Beane,  1990).  As  a result,  the  affective  curriculum  does  not  fully  encompass  the  scope  of  the  affective 
domain  of  learning,  but  often  emphasizes  the  teaching  of  only  certain  dimensions  (such  as  the  “moral”), 
while  paying  little  overt  attention  to  other  dimensions  (such  as  the  “emotional”). 

In  order  to  better  design  and  develop  curricula  and  instructional  methods  in  the  affective  domain 
of  learning,  Martin  & Reigeluth  (1999)  divide  it  into  six  dimensions:  emotional  development,  moral 
development,  social  development,  spiritual  development,  aesthetic  development,  and  motivational 
development.  Each  of  these  six  dimensions  is  associated  with  its  own  unique  components  of  instructional 
value,  such  as  knowledge,  skills,  attitudes,  etc.  (Definitions  and  associated  components  for  each  dimension 
are  tabulated  in  figure  1 and  figure  2.)  Clearly,  the  fact  that  each  of  these  dimensions  has  its  own  unique 
criteria  for  consideration  in  instructional  sequencing  and  instructional  methods  poses  difficulties  for  anyone 
who  wishes  to  design  instructional  strategies  foruse  within  the  affective  learning  domain.  Because  ofspace 
limitations,  1 will  restrict  my  discussion  of  the  reasons  for  this  state  of  affairs  to  the  latter  part  of  this  paper. 
The  major  focus  of  this  paper  will  be  on  the  question  of  sequencing  in  the  development  of  emotional 
learning.  This  topic  is  very  seldom  discussed.  However,  with  the  recognized  importance  of  emotional 
education  in  present  day  society  (Goleman,  1995),  there  exists  an  urgent  need  to  design  and  develop 
emotional  literacy  learning  soon,  so  that  educational  methods  to  develop  an  emotionally-mature  personality 
can  be  implemented  starting  at  the  early  stages  of  childhood. 


BEST  COPY  AVAIL/  D' 


Term 

Definition 

Emotional 

Development 

Understanding  your  own  and  others’  feelings  and  affective  evaluation,  learning  to  manage 
those  feelings,  and  wanting  to  do  so. 

Moral 

Development 

Building  codes  of  behavior  and  rationales  for  following  them,  including  developing 
prosocial 

attitudes,  often  in  relation  to  caring,  justice,  equality,  etc. 

Social 

Development 

Building  skills  and  attitudes  for  initiating  and  establishing  interactions  and  maintaining 
relationships  with  others,  including  peers,  family,  coworkers,  and  those  different  form  ours. 

Spiritual 

Development 

Cultivating  an  awareness  and  appreciation  of  one’s  soul  and  its  connection  with  others’ 
souls, 

with  God,  and  with  all  His  Creation. 

Aesthetic 

Development 

Acquiring  an  appreciation  for  beauty  and  style,  including  the  ability  to  recognize  and  create 
it;  commonly  linked  to  art  and  music,  but  also  includes  the  aesthetics  if  ideas. 

Motivational 

Development 

Cultivating  interests  and  the  desire  to  cultivate  interests,  based  on  the  joy  or  utility  they 
provide,  including  both  vocational  and  avocational  pursuits. 

Fig.  1 . Definitions  of  the  dimensions  of  affective  development  (from  Martin  & Reigeluth,  1 999,  p.  494) 


COMPONENTS  OF  INSTRUCTIONAL  VALUE 

DIMENSIONS 

Knowledge 

Skills 

Attitudes 

Others? 

Emotional 

Development 

Knowing  that  others 
experience  the  same 
emotions  you  do,  such 
as  joy  and  anger 

Recognizing 
emotions 
Controlling  one’s 
emotions 

I want  to  be  happy. 

I don’t  like  to  be  angry. 

? 

Moral 

Development 

Understanding  moral  & 
ethical  rules  of  the 
culture,  such  as  caring, 
justice,  equality 

Moral  reasoning 
skills 

Problem-solving 
skills  in  the  realm  of 
morals 

I want  to  be  honest. 

I am  in  favor  of  having 
ethical  standards. 

? 

Social 

Development 

Understanding  group 
dynamics  and  democratic 
ideas,  such  as  the  role  of  a 
facilitator 

Social  skills, 
including 
interpersonal 
communication  skills 

I want  to  interact 
positively  with  others. 

I am  opposed  to 
resolving  disagreements 
by  Fighting. 

? 

Spiritual 

Development 

Knowledge  of  religious 
precepts  about  the 
spiritual  world,  such  as 
the  nature  of  the  soul 

Skills  for  getting  in 
touch  with  your  inner 
self 

Ability  to  love  others 
selflessly 

I want  a spiritual  life. 

I am  in  favor  of  prayer 
to  build  a relationship 
with  God. 

? 

Aesthetic 

Development 

Understanding  the 
subjective  nature  of 
aesthetics,  such  as  the 
relationship  between 
one’s  values  and  one’s 
judgement 

Skills  for  assessing 
aesthetic  qualities 
Skills  for  generating 
aesthetic  creations 

I want  to  surround 
myself  with  things  of 
beauty. 

I appreciate  an  elegant 
theory. 

? 

Motivational 

Development 

Understanding  internal 
and  external  rewards  for 
sustained  activity,  such  as 
joy  and  sense  of 
accomplishment 

Skills  for  developing 
one’s  interests,  both 
immediate  and  life- 
long 

I want  a career  that  I 
enjoy. 

I am  opposed  to 
hobbies  related  to  guns. 

? 

Fig.  2.  A conceptual  model  for  affective  development  (from  Martin  & Reigeluth,  1999,  p.  493) 


The  paper  will  begin  with  a discussion  of  the  concept  of  “sequencing”  as  it  applies  to  curriculum 
development.  It  will  then  proceed  to  a more  detailed  discussion  of  sequencing  strategies  in  the  emotional 
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literacy  curriculum.  In  order  to  arrive  at  suggested  guidelines  for  sequencing  strategies  for  the  emotional 
literacy  curriculum,  I will  use  PATHS,  an  emotional  literacy  curriculum  program,  as  an  exemplary  case  for 
analysis.  After  the  analysis  and  discussion  of  PATHS,  as  well  as  of  other  literature  in  emotional 
development,  I will  synthesize  a set  of  guidelines  for  emotional  learning  sequencing  strategies,  and  explore 
their  further  implications  for  the  design  and  development  of  emotional  learning. 


2.1.  “Sequencing”  in  Curriculum  and  Instruction 

There  are  several  crucial  decisions  to  make  when  designing  and  developing  valid,  worth-while 
curricula  and  instructional  methods.  These  decisions  include  selecting  educational  objectives,  selecting 
learning  experiences,  organizing  learning  experiences,  and  evaluating  learning  experiences  (Tyler,  1 949). 
Tyler  goes  on  to  cite  three  basic  guiding  criteria  to  be  applied  in  the  organization  of  learning  experiences. 
These  are:  continuity,  sequence,  and  integration.  “Continuity  refers  to  the  vertical  reiteration  of  major 
curriculum  elements”  (p.  84);  “[sequence  . , . emphasizes  the  importance  of  having  each  successive 
experience  build  upon  the  preceding  one  but  to  go  more  broadly  and  deeply  into  the  matters  involved”  (p. 
85);  “[integration  refers  to  the  horizontal  relationship  of  curriculum  experiences”  (p.  85).  While  Tyler 
views  these  criteria  as  sufficient  for  the  building  of  an  effective  scheme  of  organization  for  learning 
experiences,  he  did  not  provide  many  further  guidelines  for  “sequencing”  in  subsequent  discussions.  Yet 
this  aspect  of  learning,  referring  as  it  does  to  the  “increasing  breadth  and  depth  of  the  learner’s 
development”  (p.  96),  is  undeniably  of  central  importance  in  practical  curriculum  design. 

According  to  Reigeluth  (in  press),  “sequence,”  in  brief,  deals  with  “how  to  group  and  order  the 
content.”  And  the  main  factors  which  govern  this  grouping  and  ordering  lie  in  the  “relationships”  within  the 
content.  Up  to  the  present,  there  are  four  most  popular  sequencing  strategies  in  instructional  design,  each  is 
only  concerned  with  a single  type  of  relationship  within  the  content.  They  are  chronological  sequence, 
procedural  sequence,  hierarchical  sequence  and  simplifying  conditions  sequence.  “A  chronological 
sequence  is  based  on  the  temporal  order  of  events”  (Leshin,  Pollock  & Reigeluth,  1992,  p.  81).  “A 
procedural  sequence,  ...  is  based  upon  the  relationship  of  ‘order  of  performance’  of  the  steps  in  the 
procedure.  A hierarchical  sequence  is  based  upon  the  relationship  of  learning  prerequisites  among  the 
various  ski  11  s and  subskills  that  comprise  a task.  And  the  ‘simplifying  conditions’  sequence  isbasedupon 
the  relationship  of the  degree  of  complexity  of  different  versions  of  a complex  task”  (Reigeluth,  in  press). 

2.2.  Sequencing  Strategies  in  Affective  Learning 

One  thing  that  should  be  mentioned  here  is  that  these  sequencing  strategies  are  customarily 
applied  to  only  two  of  the  three  domains  of  learning:  cognitive  and  psychomotor  learning.  Sequencing 
strategies  pertinent  to  affective  learning  have  been  neglected.  Few  researchers  and  writers  have  been 
willing  to  tackle  them.  Even  the  landmark  treatise  touching  on  affective  learning,  The  Affective  and 
Cognitive  Domains:  Integration  for  Instruction  and  Research,  by  Martin  and  Briggs  (1986),  does  not 
fully  address  “sequence”  nor  does  it  shed  light  on  the  principles  and  guidelines  for  sequencing  in  affective 
learning.  However,  this  does  not  mean  that  “sequence”  in  affective  learning  is  unimportant.  On  the 
contrary,  this  has  been  the  focus  of  growing  attention  in  very  recent  years,  concerned  with  an  increased 
acknowledgment  of  the  importance  of  affective  learning.  For  example,  Dr.  Charles  M.  Reigeluth  in  the 
field  of  instructional  systems  technology  has  been  endeavoring  to  find  valid,  proven  guidance  for  making 
decisions  about  sequencing  in  affective  learning,  with  a special  focus  on  emotional  literacy  learning.  It  is 
also  under  his  encouragement  and  direction  that  the  author  is  devoting  research  to  this  topic.  The  author 
may  seem  rash  in  making  such  a commitment— assuming  a burden  which  others  have  thus  far  been  reluctant 
to  take  on.  There  must  be  thorny  issues  in  the  process,  so  the  mystery  of  sequencing  in  affective  learning 
has  not  been  explored.  What,  then,  are  the  problem(s)  or  “issues”  encountered  in  this  domain? 

Because  of  the  incompleteness  of  the  body  of  literature  and  research  directly  investigating  the 
patterns  of  sequencing  in  this  domain,  the  author  undertook  an  analysis  of  an  emotional  literacy  curriculum, 
PA  THS  (Promoting  Alternative  THinking  Strategies),  in  orderto  infera  set  of  pragmatic  guidelines  for 
sequencing,  and  factors  for  influencing  the  sequencing.  Hopefully,  this  data  can  begin  to  provide  principles 
and  guidelines  in  sequencing  strategies  for  further  design  and  development  on  affective  education. 

3.1.  Emotional  Development 

“Emotional  development”  in  short,  is  “Understanding  [how  to]  control  one’s  feelings,  and  learning 
to  manage  one’s  emotions”  (Martin  & Reigeluth,  1999).  As  stated  previously,  this  dimension  of  the 
affective  domain  of  learning  already  exists  as  a hidden  curriculum,  and  has  been  taught  implicitly. 
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Unfortunately  this  minimal  curriculum  design  paradigm  has  proven  incapable  of  meeting  the  demands 
posed  by  the  mass  of  family  and  societal  problems  caused  by  emotional  disequilibrium.  The  high  degree  of 
serious  emotional  disequilibrium  in  today’s  families  and  societies  is  surely,  to  some  extent,  a reflection  of 
the  large  quantity  of  sad  and  violent  crime  events,  emotional  ineptitude,  emotional  abuse,  desperation,  and 
recklessness  reported  in  any  day’s  newspaper  (Goleman,  1995).  In  such  an  environment,  children  are  and 
will  be  the  victims.  As  Daniel  Goleman  (1995),  a Harvard  psychologist,  states,  “[a]  massive  survey  or 
parents  and  teachers  shows  a worldwide  trend  for  the  present  generation  of  children  to  be  more  troubled 
than  the  last:  more  lonely  and  depressed,  more  angry  and  unruly,  more  nervous  and  prone  to  worry,  more 
impulsive  and  aggressive”  (p.  xiii). 

This  emotional  crisis  has  been  expounded  rather  clearly  in  Goleman’s  popular  treatise,  Emotional 
Intelligence : why  it  can  matter  more  than  IQ  ( 1 9951.  Thi  s book,  which  bases  its  conclusions  on  studies  of 
the  brain’s  structure  and  function,  and  draws  on  the  related  fields  of  physiology,  neurology,  psychology  and 
behavior,  etc.,  has  revealed  much  about  how  the  human  “emotional  mind”  functions  and  its  significance  in 
directing  human  behavior  throughout  life.  Goleman  points  out  that  the  factors  which  lead  to  success  in  life 
and  career  involve  not  only  the  Intelligence  Quotient  (IQ)  but  also  a more  dominant  index:  the  Emotional 
Quotient  (EQ).  Goleman  states  that  the  emotional  quotient  represents  self-awareness  and  impulse  control, 
persistence,  zeal  and  self-motivation,  empathy  and  social  deftness.  He  further  indicates  that  an  individual 
with  higher  emotional  intelligence  tends  to  have  better  opportunities  and  greater  success  in  any  field. 

Research  has  shown  that  emotional  development  occurs  ahead  of  cognitive  development 
(Greenberg  & Kusche,  1993).  It  is  further  suggested  that  EQ  is  not  genetically  determined,  but  can  be 
nurtured  and  strengthened  for  years  after  birth  by  emotional  literacy  programs  designed  to  enhance 
emotional  intelligence.  Goleman  urged  that  our  schools  should  have  emotional  curricula  to  prepare  our 
young  for  successful  life.  He  outlined  several  vital  curricula  on  emotional  literacy  for  elementary  students, 
designed  to  direct  them  to  manage  their  feelings  and  solve  daily  interpersonal  conflicts.  Among  these  the 
one  most  suitable  for  my  project,  and  hence  chosen  here  as  a case  for  study,  is  the  PATHS  curriculum, 
developed  by  Greenberg,  Kusche,  and  associates. 

3.2.  The  PATHS  Curriculum 

The  Purpose  of  PATHS 

“The  purposes  of  The  PATHS  Curriculum  are  to  enhance  social  and  emotional  competence  and 
understanding  in  children,  as  well  as  to  develop  a caring,  prosocial  context  that  facilitates  educational 
processes  in  the  classroom  (The  PATHS  instructional  manual,  p.  1)”.  This  is  an  experimental-based 
program  initially  designed  for  deaf  children.  The  original  title  is  called  Promoting  Social  and  Emotional 
Development  in  Deaf  Children:  The  PATHS  Project  (Greenberg  & Kusche,  1993).  Since  1982  this 
program  has  been  revised  and  expanded  to  meet  the  needs  of  different  types  of  children  and  of  multi- 
culture. This  program  is  designed  for  kindergarten  through  6.  It  can  not  only  serves  as  a intervention 
program  for  children  with  physical  or  mental  or  cognitively  delayed  or  severe  behavioral  disturbance  or 
emotional  problems,  but  also  can  serve  as  a prevention  program  for  normal  or  regular  or  even  gifted 
children.  And  this  program  has  been  successfully  applied  in  all  above  different  types  of  children. 

PATHS  can  be  effective  as  both  a prevention  and  as  an  intervention  program  (Greenberg  & 
Kusche,  1993).  “These  dual  functions  are  especially  of  practical  value  to  educators,  since  today’s 
classrooms  generally  include  a mixture  of  children  who  are  in  need  of  intervention  as  well  as  children  who 
are  not:  at  risk,”  but  who  can  nevertheless  benefit  from  prevention  programs  designed  to  reinforce  healthy 
functioning.  We  have  found  PATHS  to  be  useful  with  a variety  types  of  children”  (The  PATHS 
instructional  manual,  p.  2). 

Theoretical  Roots 

The  design  of  PATHS  takes  into  account  aspects  of  diverse  theories  of  human  behavior  and 
development.  According  to  the  authors,  these  theories  include  developmental  social  cognition  (e.g., 
Greenspan  [1981]  and  Shantz  [1983]),  cognitive  developmental  theory  (e.g.,  Dewey  [1894,  1933]  and 
Piaget  [1981  ]),  psychoanalytic  developmental  psychology  (e.g.,  Freud  [1981],  Nagera  [ 1 966],  Pine  [ 1 985]), 
and  attachment  theory  (e.g.,  Bowlby  [1973,  1982]),  interpersonal  development  (e.g.,  Selman  [1980]), 
interpersonal  problem  solving  (Spivak,  Platt  & Shure  [1976]),  moral  development  (e.g.,  Kohlberg  [1980]), 
cognitive-social  learning  (e.g.,  Bandura  [1986]),  cognitive-behavior  therapy  (e.g.,  Kendall  & Braswell 
[1985],  Meichenbaum  [1977]),  Bretherton  [1985],  Main  et  al.  [1985]),  and  so  on  (Greenberg  & Kushe, 
1993). 
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These  above  research  studies  have  been  integrated  into  the  so-called  Affective-Behavior- 
Cognitive-Dynamic  (ABCD)  theoretical  model  of  development,  which  forms  the  foundation  PATHS.  The 
ABCD  model  emphasizes  the  dynamic  relationship  among  affect,  behavior,  and  cognition,  integrating  these 
three  aspects  so  as  to  facilitate  children’s  positive  and  healthy  personality  development  and  social 
functioning  throughout  their  different  developmental  phases  (Greenberg  & Kushe,  1993). 

Learning  Goals 

PATHS  addresses  the  following  goals  in  the  areas  of  social  and  emotional  development: 

L Increased  self-control. 

2.  Enhanced  self-esteem,  self-confidence,  and  the  ability  to  give  and  receive  compliments. 

3.  Increased  understanding  and  use  of  the  vocabulary  of  emotions,  verbal  mediation,  dialoguing,  and 
interpersonal  communication. 

4.  Improved  ability  to  recognize  and  interpret  the  differences  between  feelings,  behaviors,  and 
perspectives  of  self  and  others. 

5.  Understanding  of  attributional  processes  that  lead  to  an  appropriate  sense  of  self-responsibility. 

6.  Recognition  and  understanding  of  how  one’s  behaviors  affect  others. 

7.  Enhanced  motivation  and  use  of  creativity. 

8.  Increase  understanding  and  use  of  logical  reasoning  and  problem-solving  vocabulary. 

9.  Improved  knowledge  of,  and  skill  in,  the  steps  of  social  problem-solving:  leading  to  the  prevention 
and/or  resolution  of  problems  and  conflicts  in  daily  life. 

(The  PATHS  Instructional  manual,  p.  2) 

These  nine  goals  are  designed  to  develop  the  child’s  self-control,  positive  self-esteem,  emotional 
awareness  and  management,  and  interpersonal  problem-solving  skills.  They  are  quite  inclusive,  and  are 
consistent  with  the  essential  components  of  the  current  dominant  conceptual  models  or  frameworks  for 
emotional  intelligence.  PATHS  comprises  one  hundred  and  thirty-one  lessons,  each  lesson  taking  about  20- 
30  minutes  of  classroom  time.  These  lessons  are  classified  into  four  major  units:  readiness  and  self-control 
(1  volume),  feelings  and  relationships  (3  volumes),  problem-solving(l  volume),  and  supplementary  lessons 
(1  volume).  These  units  deal  with  five  major  conceptual  domains:  self-control,  emotional  understanding, 
building  self  esteem,  relationships,  and  interpersonal  problem-solving  skills. 

Philosophy  Underlying  the  Goals 

The  philosophy  intrinsic  to  PATHS  reflects  the  authors’  belief  in  educating  the  whole  child,  which 
is  compatible  with  John  Dewey’s  philosophy  of  wholistic  education  (The  PATHS  Instructional  manual,  p. 
8).  The  wholistic  approach  to  education  entails  treating  language,  cognition,  memory,  emotion,  and 
behavior  as  intimately  interrelated  and  all-important  aspects  of  every  chi  Id’s  personality.  In  accord  with  this 
philosophy,  the  authors  affirm  that 

Emotions  affect  all  ofus  on  a daily  basis  throughout  our  lives.  Understanding  and  dealing  with  our 
feelings  and  those  of  others  are  therefore  areas  that  will  be  of  continual  concern  to  all  of  us, 
whether  or  not  we  are  aware  of  this.  Understanding  emotions  often  becomes  more  complex  as  we 
get  older.  Further,  sharing  emotional  issues  with  others  continues  to  be  at  least  one  of  the  major 
motivations  for  social  interaction  and  is  often  the  “glue”  for  intimacy  and  friendship  (Youniss, 
1980;Selman,  1980).”  (Instructional  Manual,  p.  125) 

Unfortunately,  traditional  education  has  emphasized  the  acquisition  of  cognitive  skills  much  more 
than  that  of  aspects  of  emotional  cultivation,  such  as  emotional  awareness,  emotional  control,  and  so  on. 
Thus,  children  have  never  been  taught  through  a systematic  emotional  development  curriculum,  and 
teachers,  more  familiar  with  instructional  methods  in  cognitive  learning  than  with  those  for  emotional 
learning,  have  less  confidence  in  teaching  emotional  learning.  Since  PATHS  provides  a complete 
curriculum  and  more  than  one  hundred  lesson  plans  on  emotional  literacy  for  elementary  school,  it  may 
offer  elementary  teachers  anew  level  of  confidence  and  comfort  in  thinking  about  wholistic  education  and 
teaching  in  a wholistic  way. 

Instructional  Methods 

PATHS  is  intended  to  be  a separate  course  within  the  general  curriculum,  and  is  suited  to  the  k-6 
elementary  years.  It  employs  a variety  of  instructional  methods.  Because  it  concentrates,  not  on  teaching 
cognitive  skills,  but  rather  on  cultivating  affect,  it  combines  a variety  of  instructional  methods  so  as  to 
promote  progressive  emotional  and  behavioral  change  in  the  individual.  These  instructional  methods 
include  dialoguing,  role-playing,  story-telling,  simulation,  modeling,  social  and  self-reinforcement, 
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attribution  training,  and  verbal  mediation.  Visual,  verbal,  and  kinesthetic  modalities  are  combined  to 
promote  learning.  These  instructional  methods  are  mostly  intended  to  be  conducted  by  the  teachers; 
however,  parents  are  also  expected  to  participate  in  modeling  outside  the  school  environment. 

The  Significance  of  PATHS 

The  PATHS  program  has  achieved  improvements  in  several  areas  of  children’s  learning. 
Goleman  notes  (1995,  p.  306),  that  the  PATHS  curriculum  has  been  successful  in  the  following  areas: 

• Improvement  in  social  cognitive  skills 

• Improvement  in  emotion,  recognition,  and  understanding 

• Better  self-control 

• Better  planning  for  solving  cognitive  tasks 

• More  thinking  before  acting 

• More  effective  conflict  resolution 

• More  positive  classroom  atmosphere 

For  the  purpose  of  this  project,  only  the  Feelings  and  Relationships  Unit  is  included  for 
investigation  because  it  represents  the  major  portion  and  thrust  in  emotional  literacy. 

4.1.  Micro-Level  (Within-Lesson)  Sequencing  Strategies  in  PATHS 

Basically,  the  pattern  of  sequencing  within  each  lesson  of  the  Feelings  and  Relationships  Unit 
follows  seven  steps  set  out  by  the  developers  of  PATHS  (PATHS,  v.  1 , p.  xxix).  Figure  3 shows  these  seven 
steps  (PATHS,  v.  1 , p.  xxix),  as  well  as  an  example  of  the  script  of  a lesson  on  HAPPY  (PATHS,  v.  1 , pp. 
43-44). 


Step  of  a Lesson 

Script  of  a Lesson  on  HAPPY 

1.  Presenting  a given  emotion 
with  a simple  definition. 

Happy  is  the  way  we  feel  when  we  are  glad  about  something  or  when  we  enjoy 
something. 

2.  Showing  pictures  of  the 
emotional  expression. 

(Take  the  Happy  Feeling  Face  out  of  the  Feelings  Chart  and  show  it  to  the  class.) 
This  Felling  Face  shows  someone  who  is  feeling  happy.  Do  you  think  that  happy 
feels  comfortable  or  uncomfortable  inside? 

3.  Providing  examples  of 
situations  that  typically  elicit  the 
feeling. 

We  can  feel  happy  for  a lot  of  reasons.  There  are  lots  of  times  we  feel  happy,  like 
if  we  get  to  go  some  place  we  really  like,  if  we  get  to  play  with  our  friends,  if  we 
get  something  really  special,  and  so  on. 

4.  Having  the  children  talk  about 
times  when  they  have 

experienced  the  emotion. 

Can  anyone  think  of  a time  when  they  felt  happy? 

5.  Modeling  and  labeling  the 
feeling  as  a group. 

Now  let’s  look  at  two  photographs  of  people  who  feel  happy. 

(Point  out  the  features  that  indicate  happiness. ..Model  as  needed  for  further 
clarification  or  demonstration.) 

6.  Having  the  children  make 
“Feeling  Faces”. 

Now  let’s  all  practice  the  word  HAPPY  together.  I want  all  of  you  to  think  about 
how  you  feel  when  you  feel  happy.  Try  to  make  your  body  feel  happy  and  make 
your  face  look  happy.  Try  to  feel  happy  inside. 

7.  Providing  the  children  with  an 
activity  through  which  the 
emotion  can  be  experienced  on  a 
more  personal  level. 

Then,  while  you  are  feeling  happy,  I want  all  of  you  to  say  the  word  HAPPY 
together.  Then  we  will  all  spell  it  together.  Then  we  will  say  it  and  spell  it  again. 
After  that,  we  will  say  the  sentence  “I  feel  happy”  together.  If  you  need  to 
remember  how  to  spell  the  word  happy,  look  up  at  the  Feelings  Chart  or  the  board 
(or  overhead)  where  I have  written  the  word.  Is  everybody  ready?  Good.  Let’s  all 
look  and  feel  HAPPY  and  practice  the  word  together. 

Fig.  3.  Seven  steps  of  a lesson  and  the  script. 


At  first  glance,  the  seven  steps  of  PATHS  may  seem  superficial  and  oversimplified.  However,  by 
deeply  pondering  on  these  seven  steps,  it  is  apparent  that  they  show  an  interesting  model  of  sequencing  of 
content  in  emotional  literacy.  The  seven  steps  are  converted  to  the  following  analysis  (see  Figure  4): 


Step  of 

Domain  of 

Guideline  of  Sequencing 

Sequencing  Strategies 

Learning 

Strategies 
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1.  Presenting  a given  emotion 

1.  Concept  generalization. 

with  a simple  definition. 

2.  Showing  pictures  of  the 

(Cognitive) 

2,  Concept  classification. 

emotional  expression. 

3.  Providing  examples  of 

(Cognitive) 

3.  Principle  learning. 

Cognitive-to-Behavior-to- 

situations  that  typically  elicit 
the  feeling. 

4.  Having  the  children  talk  about 

(Cognitive) 

4.  Cognitive  & emotional. 

Affective 

times  when  they  have 
experienced  the  emotion. 

5.  Modeling  and  labeling  the 
feeling  as  a group. 

6.  Having  the  children  make 
“Feeling  Faces”, 

7.  Providing  the  children  with  an 
activity  through  which  the 
emotion  can  be  experienced  on 
a more  personal  level. 

5.  Cognitive  & emotional. 

6.  Behavioral. 

7.  Emotional  & empathy. 

Extemal-to-Intemal 

Fig.  4.  Micro-level  sequencing  strategies  in  PATHS. 


Step  1 gives  a definition  of  one  kind  of  emotion  which  seems  to  deal  with  the  concept 
generalization.  Pictures  of  facial  expressions  shown  in  step  2 depict  the  concept  classification.  Therefore, 
both  these  first  two  steps  are  aimed  at  the  cognitive  aspect  of  learning,  which  strives  to  recognize, 
understand  and  even  discriminate  between  and  among  the  concepts  of  certain  feelings.  In  step  3,  kinds  of 
situations  eliciting  the  feelings  are  shown  to  point  out  that  there  is  a cause  and  effect  relationship  between 
the  two.  This  is  dealing  with  principle  learning.  Each  of  these  three  steps  is  concerned  primarily  with  the 
cognitive  domain  of  learning,  which  “deal[s]  with  the  recall  or  recognition  of  knowledge  and  the 
development  of  understandings  and  intellectual  abilities  and  skills”  (Reigeluth  & Moore,  1999.  P.  52). 

In  step  4,  the  learners  must  talk  about  times  when  they  have  experienced  the  feelings  already 
conceptualized  and  situated  (by  example).  The  learner  is  called  upon  to  reach  into  his/her  personal  feelings 
and  connect  these  concepts  and  practices  to  the  actual,  felt  emotion.  Although  this  is  still  cognitive- 
oriented,  it  goes  to  a deeper  emotional  level  of  learning. 

In  step  5,  modeling  and  labeling  the  feelings  enables  learners  to  recognize  the  feeling  through  a 
cognitive-based  approach.  This  combines  the  cognitive  and  affective  components.  In  step  6,  having  the 
children  make  “Feeling  Faces”  entails  asking  the  learners  to  exhibit  the  emotions  as  if  they  actually  were 
feeling  them.  They  are  asked  to  show  these  feelings  by  their  facial  expressions,  and  further,  they  are  asked 
to  imagine  the  feelings  as  they  do  this.  This  requires  them  to  emphasize  the  feeling  beyond  the  cognitive- 
based  learning.  This  combines  the  behavioral  and  affective  components  (a  sense  of  empathy).  Step  7 
deals  with  a specific  individual  situation  for  feedback  on  how  each  learner  would  react  emotionally.  An 
example  would  be  to  ask  “How  would  you  feel  if  your  money  were  stolen?”  This  level  asks  the  learners  to 
internalize  the  situation  and  to  simulate  their  own  emotional  reaction.  This  brings  to  awareness  as  closely 
as  possible  the  feelings  available  to  each  of  the  learners. 

In  conclusion,  in  analyzing  the  seven  steps  of  sequencing  within  a lesson,  it  becomes  apparent  that 
the  within-lesson  sequence  is  mainly  structured  so  as  to  proceed  from  cognitive  to  behavioral  to  affective, 
and  from  external  to  internal  through  recognition,  reflection  and  ultimately  to  the  empathetic-(emotional)-- 
state  which  is  feeling.  These  sequencing  strategies  are  synthesized  in  figure  4. 

4.2.  Factors  Affecting  Within-Lesson  Sequencing  Strategies  in  PATHS 

The  author  has  been  endeavoring  to  uncover  the  rationale,  or  theoretical  roots,  embedded  within 
the  practice  of  the  within-the-lesson  sequencing  strategies  shown  above.  It  has  been  found  that  the 
arrangement  of  within-lesson  sequencing  strategies  is  consistent  with  the  following  rationale  and  practice: 

According  to  Kusche,  Between  ages  five  and  seven,  there  are  several  changes  in  emotional 
development,  including: 

• the  spontaneous  generation  of  emotional  concepts 
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• identification  of  anger 

• understanding  cause-effect  relationships 

• emotional  perspective-taking 

• recognition  of  emotional  facial  expressions 

• internal-external  generation  of  emotional  concepts. 

(Personal  communication,  November  1997) 

Therefore,  these  steps  in  developmental  readiness  correspond  to  these  sequencing  strategies.  Most 
of  these  seven  steps  of  sequencing  strategies  are  designed  to  correspond  with  developmental  readiness. 
During  this  developmental  stage,  language  is  also  believed  to  provide  three  ways  to  facilitate  the  child’s 
behavioral  and  emotional  control  (Greenberg  & Kusche,  1993,  p.  77): 

“First,  it  serves  to  communicate  one’s  internal  states  to  others...  Second,  language  provides  an 
internal  executive  function  that  can  mediate  between  impulse  and  behavioral  action...  Finally, 
language,  and  possibly  other  forms  of  symbolic  representation,  allow  the  child  to  become 
consciously  aware  of  his  or  her  feelings.” 

One  method  of  facilitating  a child’s  verbal  and  emotional  control  is  verbal  labeling  of  emotional  states, 
which  help  the  child  develop  powerful,  new  forms  of  self-control  and  self-expression. 

Also  between  the  ages  offive  and  seven,  the  child’s  teacher,  parents,  and  other  adults  play  very 
important  parts  as  role  models,  demonstrating  ways  of  using  cognitive  and  affective  processes  for 
managing  frustration,  maintaining  control  during  times  of  emotional  turmoil,  and  dealing  with  interpersonal 
conflict.  This  would  benefit  the  child’s  emotional  development  and  social  competence. 

5.1.  Macro-Level  Sequencing  in  PATHS 

Macro-level  sequencing  analysis  deals  with  how  all  the  topics  to  be  taught  in  the  entire  curriculum 
(i.e.,  here,  the  Feelings  and  Relationships  Unit  of  PATHS)  ought  to  be  sequenced.  Since  the  entire  unit 
covers  fifty-six  lessons  of  several  topics,  it  is  not  easy  at  the  beginning  to  figure  out  thoroughly  all  kinds  of 
sequencing  to  be  applied,  especially  in  an  implicit  way.  Therefore,  the  author  begins  with  the  analysis  of 
the  sequencing  patterns  in  the  unit,  to  see  ifthey  follow  in  a topical  or  a spiral  sequence.  This  isnotaneasy 
task  because  the  connotations  of  the  families  of  emotional  feeling  words  are  controversial,  especially  those 
which  pertain  to  more  complex  feelings.  Several  schools  of  research  on  emotion  have  already  created 
families  of  classifications.  These  share  some  similarities,  but  the  differences  have  created  some 
considerable  controversy  (Goleman,  1 995).  This  may  produce  various  interpretations  of  the  sequencing  at 
the  macro-level  content.  Unfortunately,  the  developers  of  PATHS  do  not  fully  elaborate  on  the 
classification  of  families  of  emotions,  but  simply  separate  all  feeling  words  into  “comfortable”  and 
“uncomfortable”  feelings  for  the  convenience  of  children  to  relate  to  them.  Therefore,  the  author  of  this 
paper  had  to  delineate  less  generally  to  achieve  a more  complete  classification  of  words. 

To  begin  classification,  the  author  first  reviewed  the  content  of  each  PATHS  lesson  carefully  to 
understand  the  meaning  of  the  feeling  words  in  the  context  of  the  lessons.  This  process  helped  the  author  to 
arrive  at  a more  distinct  classification  of  many  of  the  words.  However,  the  remaining  words  were  fuzzy  in 
personal  interpretation,  vague,  and  complex  in  nature.  Therefore,  to  achieve  a better  approach,  the  author 
has  consulted  subject  matter  experts  and  academic  research  materials  on  emotions  (Goleman,  1995;  Ortony, 
Clore  & Collins,  1988,  p.  27).  Finally,  a classification  figure  was  produced:  Genera  of  Feelings  & 
Relationships  in  PATHS  (see  Figure  5.).  Because  the  Feelings  and  Relationships  Unit  covers  not  only 
feeling  words  but  also  other  activities  involving  behavior  management  in  the  domain  of  emotional  literacy, 
the  classification  figure  extends  beyond  feeling  words.  This  classification  helped  the  author  name  the  topic 
of  each  lesson.  This  made  it  easier  for  later  analysis  of  the  sequencing  pattern  to  which  lessons  pertain. 


Feeling 

A general  introduction  to  emotional  state 

Enjoyment 

Happy,  delighted,  proud,  content,  satisfied 

Sadness 

Sad,  a little  down,  depressed,  (disappointed,  hopeful),  lonely 

Private 

Private,  privacy,  hiding  feelings 

Fine 

(Fine,  excited,  tired) 

Anger 

Angry,  mad,  grouchy,  grumpy,  furious,  frustrated,  resentful,  anger  management 

Fear 

Afraid,  (scared,  safe),  uneasy,  terrified,  nervous,  (anxious,  calm  or  relaxed) 
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Surprise 

(surprised,  expect) 

Love 

(Love,  hate),  (like,  don’t  like),  (kind,  malicious),  (included,  rejected,  excluded), 
generous,  forgiving. 

Disgust 

Disgusted 

Shame 

Ashamed,  guilty,  shy,  embarrassed,  humiliated 

Interest 

(Curious,  interested,  bored) 

Confused 

(Confused,  worried,  sure) 

Observation 

Observing  emotional  clues  in  other  people,  observing  emotional  clues  in  ourselves 

Intentionality 

(Accident,  on  purpose) 

Manner 

Manner 

Change  Feeling 

Change  feeling 

Jealous 

Jealous  or  envious 

Selfish 

Gready,  selfish 

Note:  ( ) denotes  antonymous  feelings 

Fig.  5.  Genera  of  Feelings  & Relationships  in  the  PATHS. 


The  author  then  planed  the  order  of  each  lesson  to  figure  out  the  flow  of  the  ordering  of  the  topics. 
From  this,  a flowchart  of  lesson  ordering  was  produced  (see  Figure  7).  The  flowchart  illustrates  several 
findings  that  are  unique  in  this  field  of  knowledge  and  will  enhance  further  analysis  of  the  deeper,  implicit, 
and  more  complex  relationships  in  sequencing.  Later  parts  will  have  more  elaboration. 

5.1.1.  A spiral/topical  curriculum 

In  the  flowchart,  it  shows  that  the  patterns  of  sequencing  of  the  topics  to  be  taught  tend  to  be  a 
combination  of  two  popular  types  of  curriculum  sequencing:  spiral  and  topical  sequencing.  In  topical 
sequencing,  the  “topic  (or  task)  is  taught  to  whatever  depth  of  understanding  (or  competence)  is  required, 
before  moving  to  the  next  one.”  (Reigeluth,  1 992,  p.  2.6)  In  spiral  sequencing,  several  passes  over  the 
material  are  used  to  present  the  basics  of  each  topic,  one  at  a time.  After  the  basics  of  each  topic  are  taught, 
each  topic  is  revisited  in  greater  depth.  In  this  form  of  sequencing,  the  learner  “spirals”  through  each  topic, 
and  each  topic  is  taught  until  the  necessary  depth  is  reached  (Reigeluth,  1 992).  The  following  sections 
explain  the  functioning  of  spiral  sequencing  and  topical  sequencing  within  PATHS.  Figure  6 synthesizes 
the  findings. 


Type  of  Curriculum  Sequencing 

Guideline  of  Sequencing  Strategies 

Spiral  Sequencing 

Basic-to-complex  continuum 
Hierarchical  relationship 

Topical  Sequencing 

General-to-detailed  continuum 
Simplified  conditions  method 
Synonymous  relationship 
Antonymous  relationship 

Fig.  6.  Macro-Level  Sequencing. 


5.1.2.  Spiral  curriculum 

Before  discussing  this  section,  there  are  two  important  continua  which  need  to  be  addressed: 
simple-to-complex  continuum  and  general-to-detailed  continuum.  These  two  continua  have  been  regarded 
as  important  components  to  the  success  of  instructional  sequencing.  “The  simple-to-complex  continuum 
refers  primarily  to  a continuum  formed  by  adding  or  removing  parts  of  ideas  (either  principle  or 
procedures).”  (Reigeluth,  1983,  p.  346)  “The  general-to-detailed  continuum  refers  primarily  to  a continuum 
formed  by  subdividing  ideas  (either  concepts  or  procedures)  or  by  lumping  ideas  (subordinate  concepts  or 
subordinate  procedures)  together.  General  has  breadth  and  inclusiveness  (i.e.,  lots  of  things  lumped 
together,  whereas  detailed  usually  narrow  (subdivision)”  (Reigeluth,  1 983,  p.  344). 
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In  the  flowchart,  it  shows  that  the  lesson  sequence  is  a combination  of  both  spiral  and  topical 
patterns  of  sequencing.  The  spiral  sequencing  in  the  flowchart  is  consistent  with  the  simple-to-complex 
continuum.  Here  “simple”  denotes  in  PATHS  the  so-called  “basic”  or  “primary”  feelings,  such  as: 
happiness,  sadness,  anger,  fear,  and  so  on.  These  emotions  are  the  basis  from  which  human  beings  develop 
other  kinds  of  feelings  through  interaction  with  society.  And  there  are  more  “sophisticated”,  “educated” 
feelings  which  are  bom  from  these  basic  feelings,  and  they  are  entitled  “complex”  feelings.  So  “complex” 
denotes  here  a blend,  or  variation,  or  nuance  of  feelings  (Goleman,  1 995).  They  are  composed  of  more 
complex  ingredients  such  as  interpersonal  skills  and  life  experiences,  and  produce  the  complexity  of  guilty, 
selfishness,  and  so  on.  Therefore,  the  author  would  like  to  use  “basic-to-complex”  rather  than  simple-to- 
complex  in  this  paper  to  avoid  any  misunderstanding. 

Thus  some  lessons  follow  in  spiral  sequence,  which  means  that  the  lessons  which  treat  basic 
emotions  are  taught  ahead  of  time.  Afterthese  basic  emotions  have  been  taught,  then  those  lessons  treating 
complex  emotions  are  taught.  In  other  words,  an  understanding  of  basic  emotions  is  treated  as  a 
prerequisite  for  learning  about  complex  emotions,  and  it  is  assumed  that  basic  emotions  are  the  ones  that 
complex  emotions  are  based  upon.  Therefore,  we  may  conclude  that  a hierarchical  relationship  exists  here. 
Such  basic-to-complex,  hierarchical  relationships  can  be  seen  clearly  in  Greenberg  and  associates’  division 
of  the  fifty-six  lessons  in  the  Feelings  and  Relationships  Unit  into  4 levels  of  complexity  (PATHS,  1 994, 
v.  1,  p.xxiii-xxiiiiv).  They  are: 

Level  1:  basic  and  intermediate  emotions  (lessons  1-21.) 

Level  2:  moderately  advanced  emotions  (lessons  24-33) 

Level  3:  observations  and  manners  of  emotions  (lessons  35-47) 

Level  4:  advanced  emotions  (lessons  48-54) 

These  4 levels  of  complexity  demarcate  the  advancement  of  spiral  sequencing,  which  proceeds  through 
topics  from  level  1 to  4.  This  unique  spiral  sequencing  is  consistent  with  the  developmental  stages  of 
children  (Greenberg  & Kusche,  1 993).  The  notions  behind  the  framework  of  development  is  concerned 
with  the  learner  as  naive,  possibly  young,  and  subject  to  emotional,  physical,  and  social  maturation  during 
the  time  of  instruction. 

5.1.3.  Topical  sequencing 

Topical  sequencing  in  the  Feelings  and  Relationships  Unit  is  consistent  with  the  general-to- 
detailed  continuum  in  that  general  feelings.  For  example,  Lesson  3 : Feeling,  is  broader  and  thus  taught  first, 
while  more  detailed  feeling,  Lesson  4:  Happy,  Sad  and  Private  is  taught  later.  A lesson  of  a feeling  in 
general  may  be  adjacent  by  a lesson  (or  lessons)  of  same  feeling  in  another  but  different  “depth.”  For 
example,  lessons  35-36  teach  “observing  emotional  clues.”  Lesson  35  focuses  on  “observing  other  people’s 
emotions”,  but  lesson  8 concerns  ones’  own  emotions. 

In  topical  sequencing,  “intensity”  of  feeling  links  many  lessons;  forexample,  lessons  14-15  teach 
“fear”  with  an  increase  in  intensity  from  “scared”  to  “afraid”,  “uneasy”  to  “terrified.”  Thus,  they  are 
sequenced  according  to  the  Simplifying  Conditions  Method.  The  Simplifying  Conditions  Method  builds 
from  the  simplest  version,  instead  of  summarizing  a topic  each  time  (Reigeluth,  in  press).  For  example, 
under  the  topic  “anger”  (Lessons  7-12),  each  lesson  contains  a pure  and  focused  dimension  of  the  topic. 
Lesson  1 1 covers  only  “control  of  anger”  and  does  not  include  other  dimension,  such  as  definitions  of 
words  related  to  the  emotion. 

Synonymous  feelings  are  often  taught  together.  This  allows  the  learners  a variety  of  expressions 
for  the  same,  or  similar,  or  synonymous  feelings.  This  is  necessary  for  empathetic  communication  of 
sharing  feelings.  The  synonymous  relationship  occurs  within  one  lesson  or  in  consecutive  lessons.  For 
example,  “mad”  and  “angry,”  treated  as  synonymous  terms,  are  dealt  with  in  two  parts  of  Lesson  7. 

In  the  macro-level  of  curriculum  of  the  Feelings  and  Relationships  Unit,  feelings  of  antonymous 
relationships  often  are  taught  within  one  lesson  or  in  consecutive  lessons.  One  example  is:  in  lesson  2,  the 
feeling  “happy”  is  taught,  and  the  feeling  “sad”  is  taught  as  well.  Such  sequencing  allows  the  learner  the 
opportunity,  through  “contrast”  and  “comparison”  to  understanding  tacitly  the  difference  between  and 
among  “feelings”.  This  will  imply  a reference  for  learners  to  refer  to  later  in  life  as  to  what  “feeling’  is 
appropriate  in  a given  situation.  This  “antonymous”  relationship  becomes  a unique  characteristic  in 
emotional  literacy. 

5.  2.  Factors  Affecting  Macro-Level  Sequencing 
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5.2.1.  Spiral  curriculum 

As  noted  earlier,  emotional  development  is  restricted  by  developmental  maturation;  therefore  it  is 
not  ideal  to  teach  emotional  literacy  in  a strict  or  pure  sense  with  either  spiral  or  topical  sequencing 
patterns.  As  the  flowchart  shows,  due  to  the  level  of  developmental  maturation  on  different  stages  of 
emotional  maturation,  spiral  sequencing  should  move  to  a certain  degree  of  complexity,  pause,  and  then 
move  on  to  a certain  degree  of  detail.  Therefore,  the  basic-to-complex  continuum  becomes  a guiding 
principle  in  sequencing.  And  hierarchical  sequencing  become  necessary  when  it  is  believed  that  some  type 
of  emotion  should  be  dealt  with  as  a prerequisite  to  the  treatment  of  some  other  type  of  emotion. 

5.2.2.  Topical  sequencing 

Four  guidelines  are  apparent  in  topical  sequencing:  a general-to-specific  continuum,  the 
Simplifying  Conditions  Method,  synonymous  sequencing,  and  antonymous  sequencing.  The  nature  of 
“intensity”  in  emotions  and  the  four  components  of  the  ABCD  model  (affective,  behavior,  cognitive  and 
dynamic)  make  feelings  multiple-dimensional  for  learning.  It  is  this  implicit  complexity  that  calls  for  the 
Simplifying  Conditions  Method,  which  calls  for  sequencing  that  builds  from  the  simplest  version,  instead 
of  summarization  of  a topic  each  time.  For  example,  under  the  topic  “anger”  (lessons  7-12),  each  lesson 
contains  a pure  and  focussed  dimension  of  the  topic.  Another  example  is  that  lesson  1 1 covers  only 
“control  of  anger”  and  does  not  include  other  dimensions,  such  as  definitions  of  words  related  to  the 
emotion. 

Research  maintains  that  children  should  learn  how  to  express  the  same  state  or  feeling  with 
different  words  (Greenberg  & Kusche,  1 993).  Maybe  it  is  this  fact  about  emotional  learning  that  provides 
the  motivation  for  PATHS’s  synonymous  sequencing. 

It  is  interesting  that  emotions  are  often  contrasted  along  dichotomies;  i.e.,  comfortable- 
uncomfortable,  pleasant-unpleasant,  and  so  forth.  The  reliance  on  dichotomies  brings  about  a reliance  on 
antonymous  sequencing. 

6.  Synthesis 

The  findings  of  above  analysis  of  sequencing  in  terms  of  micro-level  lesson  and  macro-level 
curriculum  in  PATHS  have  brought  out  several  of  the  principles  that  guide  the  sequencing  of  emotional 
literacy  in  PATHS.  Figures  4 and  6 are  concise  syntheses  of  the  patterns  and  factors  influencing  the 
instructional  sequencing  in  PATHS.  We  can  see  that  this  unique  curriculum  draws  on  several  sequencing 
strategies  beyond  those  are  used  in  psychomotor  and  cognitive  learning.  These  unique  strategies  may  be 
regarded  as  reflections  of  special  factors  that  should  be  considered  when  emotional  literacy  lessons  are 
being  sequenced.  Their  appearance  here  suggests  that  it  will  not  be  appropriate,  in  curriculum  design  for 
affective  learning,  to  simply  reuse  the  sequencing  principles  proper  to  procedural  and  cognitive  learning. 
Further  guidance  from  research  and  practice  should  be  sought  in  order  to  obtain  a more  comprehensive  set 
of  guidelines  for  emotional  development  education.  Accordingly,  instructional  designers  engaging  in 
creating  instructional  sequencing  appropriate  to  an  emotional  literacy  curriculum’s  specific  demands  might 
wish  to  draw  on  the  guidelines  elaborated  here. 

7.  Conclusion 

The  purpose  of  this  paper  is  to  provide  guidance  for  emotional  literacy  learning  by  analyzing  and 
synthesizing  principles  in  the  PATHS  curriculum.  After  an  introductory  discussion  of  the  concept  of 
“sequencing”  as  it  applies  to  curriculum  development,  it  proceeds  to  a more  detailed  discussion  of 
sequencing  strategies  in  the  affective  learning  domain.  Then,  using  the  PATHS  program  as  exemplary  case 
for  analysis,  it  derives  a set  of  guidelines  for  emotional  literacy  curriculum  sequencing  strategies,  and 
explores  their  further  implications  for  the  design  and  development  of  the  emotional  literacy  curriculum. 
Although  the  above  research  has  dealt  with  only  one  dimension  of  affective  learning,  that  of  emotional 
development,  the  results  also  begin  to  illuminate  the  unique  complexity  and  potential  for  further 
development  of  the  topic  of  sequencing  in  affective  learning.  Affective  learning  is  indeed  a fuzzy  subject, 
and  has  not  yet  received  a clear  and  fully  satisfactory  definition.  However,  it  is  to  be  hoped  that  this 
author’s  efforts  and  tentative  conclusions  will  cast  sufficient  light  for  the  beginning  of  an  extended  process 
of  disclosure  concerning  the  issues  involved  with  the  sequencing  of  affective  learning. 
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Fig.  7.  Flowchart  of  lesson  ordering  in  feelings  and  relationship  units 
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Simplify  Web  Development  for  Faculty  and  Promote 
Instructional  Design 

Dr.  David  C.  Pedersen 
Embry  Riddle  Aeronautical  University 


Faculty  members  are  often  overwhelmed  with  the  prospect  of  implementing  web-based 
instruction.  In  an  effort  to  simplify  the  process  and  incorporate  some  basic  instructional  design  elements, 
the  Ed  Tech  Team  at  Embry  Riddle  Aeronautical  University  created  a course  template  for  WebCT. 
Utilizing  rapid  prototyping,  the  template  incorporates  a standardized  user  interface  tested  by  students  and 
faculty,  institutional  branding,  structured  organization,  page  starter  templates,  and  online  support  materials. 

Faculty  development  in  the  use  of  web  based  instruction  is  always  a challenge.  That  challenge  is 
multiplied  exponentially  when  those  faculty  members  are  widely  disbursed  geographically.  Embry  Riddle 
Aeronautical  University  is  unique  not  only  in  its  aviation  emphasis,  but  also  in  its  world  wide  educational 
presence.  This  paper  discusses  the  implementation  of  a course  template  to  help  simplify  the  web 
development  process  for  faculty  and  the  corresponding  training  issues  for  the  Educational  Technology 
Team. 

University  Profile 

Founded  only  22  years  after  the  historic  flight  at  Kitty  Hawk,  Embry-Riddle  Aeronautical 
University  has  achieved  recognition  as  a world  leader  in  aviation.  Each  year  thousands  of  graduates  take 
their  place  in  the  aviation/aerospace  industry  as  pilots,  aviation  managers,  aerospace  engineers,  and  other 
aviation  professionals. 

ERAU  is  unique  not  only  as  a pioneer  in  aviation,  but  also  as  a pioneer  in  distance  education.  The 
University  has  three  campuses:  two  traditional  residential  campuses  in  Daytona  Beach,  FL  and  Prescott, 
AZ  and  an  Extended  Campus  that  is  literally  worldwide.  The  Extended  Campus  offers  traditional  courses 
in  120  education  centers  located  around  the  world  and  independent  study  courses  to  students  worldwide. 

Instructional  Technology  Implementation 

Because  of  its  importance  to  aviation,  ERAU  has  placed  a high  priority  on  technology 
implementation.  For  that  reason,  faculty  members  are  highly  motivated  to  make  use  of  the  web  in  their 
classes.  Faculty  at  the  residential  campuses  and  Extended  Campus  education  centers  use  the  web  to 
enhance  their  traditional  classes.  As  such  they  need  to  learn  how  to  use  the  web  for  everything  from 
posting  syllabi  to  electronic  discussions  and  online  testing.  The  independent  studies  courses  also  make  use 
of  the  web  to  supplement  and  replace  the  traditional  paper  study  guides  and  videotapes. 

The  Challenge 

Providing  faculty  with  the  instructional  technology  training  and  support  they  need  to  utilize  the 
web  in  their  courses  is  always  a challenge.  At  ERAU  that  challenge  is  multiplied  a hundred  times  plus 
because  of  the  125  Extended  Campus  centers  and  the  two  residential  campuses.  The  task  of  training  3500 
faculty  members  is  challenging  enough,  but  when  they  are  disbursed  geographically  over  127  sites  the 
challenge  becomes  monumental. 

Course  Template 

To  help  meet  that  challenge,  the  Educational  Technology  Team  at  the  university  implemented  a 
course  template  for  WebCT.  The  rationale  for  the  template  stemmed  from  the  nearly  universal  reaction  by 
faculty  of  being  overwhelmed  by  the  web  course  development  process  and  the  general  failure  to 


127 


639 


incorporate  instructional  design  into  the  development  process.  It  was  thought  that  a course  template  would 
simplify  the  development  process  by  providing  a standardized  user  interface,  a structured  organization,  and 
page  starter  templates.  In  addition,  good  instructional  design  could  be  incorporated  and  encouraged. 

The  Template  Development  Process 

Because  of  limited  time  and  resources,  a rapid  prototyping  approach  was  utilized.  Existing 
courses  were  reviewed  to  determine  what  web  components  were  being  used  and  how  they  were  being  used. 
Students  were  surveyed  regarding  the  usefulness  of  web  components  in  their  courses.  From  that  data, 
faculty  members  who  use  the  web  were  presented  with  several  alternative  web  course  organizational 
structures.  They  were  asked  for  input  regarding  the  organization  and  the  comprehensiveness  of  the 
elements  that  were  included.  Students  were  then  presented  with  several  alternative  user  interfaces  and 
input  was  collected.  From  the  faculty  and  student  input  the  alternatives  were  narrowed  and  reviewed  again. 
A final  version  was  determined  and  field-tested. 

Template  Components 

The  user  interface  involved  the  “look  and  feel"  of  the  template,  the  elements  available  in  the 
template,  and  the  organization  and  structure  of  those  elements.  An  important  aspect  of  the  "look  and  feel" 
was  to  incorporate  institutional  branding  through  the  use  of  appropriate  graphic  elements.  The  template 
includes  all  the  elements  available  in  WebCT  even  though  most  faculty  members  do  not  use  all  of  them. 
However,  while  all  the  elements  are  included,  they  are  hidden  from  students  initially  requiring  faculty  to 
make  a conscious  decision  to  make  them  available  to  students.  The  organizational  structure  was  developed 
to  provide  logical,  efficient  access  to  the  various  components. 

To  help  jump  start  the  development  process;  page  templates  were  created  for  standard  items  such 
as  the  syllabus.  The  page  templates  are  HTML  files  that  contain  a page  structure  with  general  information 
that  can  be  edited  by  faculty  members.  Instructions  are  provided  for  downloading  the  pages,  editing  them, 
and  uploading  them.  The  inclusion  of  the  page  templates  also  made  it  possible  to  establish  links  from  the 
course  template  to  the  HTML  files  for  the  various  elements  eliminating  the  need  for  faculty  to  create  them. 

Various  instructional  design  elements  were  also  incorporated  into  the  template.  The  user  interface 
incorporated  graphic,  layout,  and  organizational  principles.  The  page  templates  incorporated  page  and 
message  design  principles.  The  primary  encouragement  towards  instructional  design  elements  was  in  the 
learning  activity  modules.  Each  module  contained  a preformatted  page  template  for  introduction,  activities, 
conclusion,  and  suggestions  pages.  Links  were  provided  in  those  pages  to  a site  with  additional 
information  explaining  the  instructional  design  techniques  that  should  be  considered. 

The  template  also  includes  a guide  for  faculty  members  on  what  is  in  the  template  and  how  to  use 
it.  In  addition,  there  is  an  online,  facilitated  course  that  takes  the  participants  through  the  complete  course 
development  process  step-by-step.  At  key  points,  they  are  asked  to  submit  portions  of  their  work  for 
review  and  feedback. 

Implementation 

All  new  WebCT  courses  are  created  using  the  template.  Faculty  members  are  not  required  to  use 
the  template.  They  can  modify  it  or  eliminate  it  and  create  their  own.  However,  the  template  provi  des  a 
starting  point,  especially  for  new  faculty  members. 

Conclusion 

A well-designed  course  template  can  simplify  the  development  process  for  faculty  members.  It 
helps  them  focus  on  the  instructional  elements  by  not  requiring  them  to  make  decisions  about  user  interface 
and  organization.  It  also  provides  them  with  a starting  point  for  developing  their  content.  The  course 
template  has  helped  the  ERAU  Ed  Tech  Team  meet  the  challenge  of  helping  faculty  utilize  the  web  for 
instruction. 
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The  Agony  and  Ecstasy  of  Digitizing  a Department’s  Faculty 

Kenneth  A.  Job 
Deborah  Lowther 
The  University  of  Memphis 


How  It  All  Started 

Over  a year  ago  the  I CL  department's  faculty  began  to  express  their  concerns  regarding  the  lack 
of  technology > usage  in  the  classroom . These  concerns  did  not  related  to  student  usage,  as  you 
would  expect,  but  to  the  faculty  themselves . Of particular  interest  was  how  the  Internet  was  or 
was  not  being  used  to  enhance  instruction . Though  the  students  are  required  to  use  various  forms 
of  technology  in  their  class  work,  faculty  involvement  was  often  limited . Dr.  Dennie  Smith,  Chair 
of the  Instruction  and  Curriculum  Leadership  (ICL)  department  spearheaded  the  faculty's  wish  to 
increase  technology > usage.  Though  the  cooperative  efforts  of  the  ICL  department  and  the  faculty^ 
of  the  Instructional  Design  and  Technology  (ID&T)  program  three  major  tasks  were  determined 
to  improve  Internet  usage  by  the  faculty . 

The  Tasks 

□ Update  and  enhance  the  current  ICL  web  site. 

□ Have  all  members  of  the  ICL  department  create  and  maintain  an  individual  web  page  with  only 
minimal  future  support. 

□ Involve  faculty  in  the  creation  of  on-line  courses. 

In  this  paper  each  of  these  three  tasks  are  explained  in  the  context  of;  what  was  done  to  achieve 
the  task,  how  the  faculty  were  involved,  and  what  insights  were  found  during  the  project. 

Task  I - Update  and  enhance  the  current  ICL  web  site. 


What  We  Did 

The  original  ICL  web  site  had  been  created  several  years  ago,  with  updates  being  limited  to  just 
contextual  changes.  It  was  decided  that  the  site  should  be  reviewed  and  enhanced  to  bring  it  in  line  with 
more  recent  web  site  designs.  Though  the  general  appearance  was  to  stay  similar  the  graphics,  navigation, 
and  features  of  the  site  were  to  be  changed.  Most  important  was  a complete  review  of  the  content  of  the 
site.  Many  of  the  text  elements  were  outdated  and  in  need  of  change,  in  order  to  reflect  the  latest 
advancements  in  the  various  ICL  programs  of  study.  The  faculty  also  requested  an  expansion  of  the  site  to 
include;  a current  events  calendar  and  on-line  forms  for  student  and  faculty  use. 

Faculty  Involvement 

Once  a course  was  determined,  the  project  was  explained  to  the  faculty  during  a general  staff 
meeting.  The  ICL  chair  impressed  upon  the  faculty,  the  need  to  be  more  technologically  active  and  how 
involvement  in  the  project  would  be  beneficial  to  the  whole  department.  The  faculty  were  asked  to 
cooperate  in  the  project  in  order  to  best  resolve  the  concerns  that  they  and  the  public  have  expressed.  It 
was  decided  to  select  a graduate  assistant  (GA),  who  was  also  a doctoral  student,  from  within  the 
department  to  help  coordinate  the  project  tasks  and  work  directly  with  the  faculty.  This  person  was  initially 
introduced  at  that  first  staff  meeting.  Later  a contact  information  sheet  was  distributed  to  the  faculty 
explaining  the  tasks  and  how  to  contact  this  GA. 

The  first  step  to  updating  the  content  was  to  meet  with  each  of  program  chairs  informally.  Each 
program;  Elementary  Education,  Special  Education,  Secondary  Education,  Reading,  Early  Childhood 
Education,  and  Instructional  Design  &Technology  had  to  be  reviewed  and  the  cooperation  of  the  faculty 
was  essential.  Once  informal  contact  was  established  and  they  understood  what  was  required,  expanded 
(easy  to  edit)  copies  of  the  web  site  content  were  distributed. 
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The  content  was  reviewed  by  the  faculty  and  return  to  the  GA  within  2-3  weeks  on  an  average. 
The  mechanical  updating  of  the  web  site  was  completed  two  weeks  later.  The  major  updates  and 
enhancements  to  the  site  were  complete  by  the  end  of  the  first  semester.  Minor  updates  are  being  continued 
each  semester  as  need  arises. 

Conclusions 

Two  factors  were  found  to  be  of  benefit  in  completing  Task  I.  First,  was  the  initial  informal 
meeting  with  the  program  chairs.  The  faculty  felt  this  meeting  made  them  a part  of  the  project  process. 
That,  their  opinions  were  of  value  and  they  were  going  to  have  an  impact  on  how  the  web  site  represented 
their  program.  Secondly,  the  use  of  expanded  paper  copies  of  the  web  site  pages  were  helpful  to  the 
faculty.  Instead  trying  to  edit  directly  from  the  computer  screen  or  use  straight  printed  copies  of  the  site 
pages,  expanded  versions  of  the  pages  were  given  to  the  faculty.  Each  web  page  was  increased  in  size  and 
given  extra  blank  spacing  for  easy  editing.  It  was  necessary  to  explain  why  the  expanded  version  was  so 
much  larger  then  the  screen  version.  Once  the  faculty  understood  that  it  was  strictly  to  ease  the  editing 
process  and  that  the  computer  screen  would  look  like  a typical  web  page  they  found  the  expanded  format 
easy  to  use. 


Task  II  - Have  all  members  of  the  ICL  department  create  and  maintain  an 
individual  web  page  with  only  minima)  future  support. 

What  We  Did 

This  particular  aspect  of  the  project  would  be  implemented  over  a two  semester  time  period. 
During  the  first  semester  the  goal  was  to  have  every  faculty  member  of  the  ICL  department  create  a 
personalized  home  page.  The  second  semester  would  be  used  for  enhancing  their  home  pages  and 
providing  refresher  instruction.  The  intention  was  that  by  the  end  of  the  second  semester  not  only  would  all 
the  faculty  have  home  pages,  but  they  would  be  able  to  maintain  them  with  only  minimal  support. 

Contacting  the  Faculty 

Four  stages  were  used  to  contact  the  faculty  in  order  to  schedule  meetings  for  creation  of  their 
home  page. 

• During  a faculty  meeting  the  task  was  explained  and  the  need  to  schedule  meetings  was 
explained.  (0%  response  by  faculty) 

• A flyer  containing  background  information  on  the  project  and  contact  information  about  the 
GA  coordinator  was  sent  all  faculty.  (20%  faculty  response) 

• A scheduling  sheet,  with  a series  of  meeting  dates,  was  sent  those  who  had  not  responded,  (an 
increase  to  80%  response) 

• Person  - to  - person  scheduling,  where  the  GA  coordinator  met  with  faculty  and  asked  for  a 
meeting  time.  (100%  response  by  faculty) 
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Conclusions  on  Contacting  the  Faculty 

It  should  be  noted  that  between  stages  two  and  three,  part  of  the  reason  for  such  a large  increase  in 
faculty  responses  was  due  to  word  of  mouth.  As  soon  as  a faculty  member  requested  a meeting  they  were 
scheduled  for  the  earliest  possible  time.  Once  these  first  sessions  were  finished  the  faculty  responded  with 
praise  for  the  work  being  done.  As  more  faculty  completed  a session  the  word  spread  around  the 
department  and  generated  more  interest.  A second  factor  in  scheduling  was  the  flexibility  of  meeting 
times.  When  dealing  with  an  audience  like  a university’s  faculty,  who  have  very  limited  time,  it  was  found 
that  a wide  range  of  meeting  times  was  responded  to  favorably.  Meetings  could  be  scheduled  from  8 -6,  M- 
F.  Also,  understanding  the  possible  need  to  rescheduling  with  little  notice  needed  to  be  considered. 

Instruction  on  Creating,  Editing  and  Publishing  Web  Pages. 

Prior  to  meeting  the  faculty  a series  of  decisions  needed  to  be  made  regarding  the  instruction  to 
take  place. 
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First,  what  approach  to  uses  for  the  instruction.  Would  it  be  One-on-One,  Small  / Large  Group 
Instruction,  and/or  Instructional  Handouts.  Prior  experiences  using  groups  or  handouts  on  campus  had  led 
to  limited  success,  so  it  was  decided  to  focus  on  a one-on-one  approach  to  the  instruction. 

Second,  was  what  type  of  software  to  use  in  creating  web  pages.  It  was  decided  to  use  Netscape’s 
Composer  for  the  following  reasons.  The  university  has  established  Netscape's  software  as  a standard  for 
all  campus  computers.  Because  of  this  no  software  had  to  be  purchased  or  installed.  The  faculty  would 
have  everything  they  needed  already  on  their  system.  Netscape's  Composer  component  works  very  similar 
to  a word  processor.  All  members  of  the  department  have  had  some  experience  with  word  processing. 
Composer  uses  not  HTML  coding.  A very  positive  response  was  expressed  by  the  faculty  to  this  news. 
Several  had  taken  classes  or  had  heard  of  HTML  and  disliked  the  idea  of  having  to  leam  it. 

Third,  setting  a time  limit  of  2-3  hours  of  instruction.  Though  the  actual  instruction  times  varied 
from  1 to  5 hours,  every  attempt  was  made  to  limit  the  instruction  time  for  improved  retention. 

Fourth,  was  the  instructional  approach.  Since  part  of  the  task's  goal  was  to  have  the  faculty 
maintain  their  pages  with  minimal  support  the  following  approach  was  used. 

1 . Use  of  pre-created  template  pages  for  a basic  backbone  structure,  that  could  be  expanded  on. 

2.  The  faculty  member  was  put  in  the  driver’s  seat.  The  instructor  provided  verbal  instructions  and 
avoided  taking  control  of  the  mouse. 

3.  Repetition  of  key  construction  steps  were  stressed  throughout  the  instruction.  For  example,  the 
creation  of  e-mail  links  occurred  three  times  and  on  three  different  pages. 

4.  Instructional  fading  was  used  as  part  of  the  learning  process.  As  each  successive  repetition  of  a 
key  construction  step  was  repeated,  the  verbal  cueing  was  decreased,  until  the  faculty  member  was 
prompted  to  try  the  step  alone.  Reassurance  was  given  prior  this  final  step  by  restating  the  fact 
that  they  would  be  warned  if  something  was  about  to  go  wrong. 

5.  Limited  handouts  were  given,  typically  in  the  form  of  a 1-2  pages  only.  The  faculty  were 
encouraged  to  take  their  own  notes. 

6.  Faculty  were  told  that  continued  support  would  be  available  after  instruction  and  at  the 
convenience  of  the  faculty. 

Fifth,  was  determining  just  what  was  to  be  taught  during  the  instructional  sessions. 

10.  Creation  and  manipulation  of  text.  Included:  fonts,  color,  size,  styles,  indenting,  alignment,  spell 
checking,  and  use  of  special  characters. 

1 1 . Insertion  and  manipulation  of  graphics  Included:  resizing,  wrapping  text  around  graphics,  and 
borders 

1 2.  Creation  of  links.  Including:  e-mail,  page-to-page,  and  page-to-web  site  links. 

13.  Creation  and  manipulation  of  tables.  Creation  of  table  was  often  left  until  the  second  semester  of 
instruction,  unless  specifically  requested  by  the  faculty. 

14.  Publishing  final  pages  to  the  university  mainframe. 
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Figure  L Example  of  Faculty  Home  Page 


What  We  are  Doing  in  Second  Semester 

The  second  semester  of  the  project  was  focused  on  refresher  instruction  for  updating  faculty  web 
pages.  Individual  sessions  are  being  scheduled  using  the  same  methods  used  in  the  first  semester. 

Current  Status 

5 . 1 5%  of  the  faculty  have  updated  their  pages  with  no  assistance. 

6.  5%  of  the  faculty  have  updated  their  pages  with  minimal  assistance. 

7.  45%  have  scheduled  or  at  least  requested  assistance. 

8.  By  the  end  of  this  semester  all  faculty  pages  are  expected  to  be  updated. 

Conclusions  on  Web  Page  Instruction 

The  faculty  responded  very  favorable  to  the  one-on-one  instruction  and  stated  they  found  it  of 
more  value  then  previous  instructional  sessions.  They  also  felt  that  by  using  this  method  of  instruction 
more  was  learned  and  would  be  retained.  The  absence  of  learning  HTML  code  was  greatly  appreciated. 
The  second  semester  seems  to  be  going  as  well  as  the  first.  So  far  all  faculty  feedback  has  been  positive 
and  it  is  expect  that  all  of  their  web  pages  will  be  updated  and/or  expanded  within  the  time  frame. 


Task  III  - Involve  faculty  in  the  creation  of  on-line  courses. 

What  We  Did 

Using  similar  contact  methods  from  Task  II  each  of  the  department's  faculty  was  contacted  and 
asked  if  they  had  an  interest  in  on-line  course  development.  Using  this  list  of  interested  faculty  we  began 
to  instruct  them  in  using  Courselnfo.  Courselnfo  is  an  Internet  based  shell  program,  that  allows  for 
creation  of  on-line  course  using  a relatively  easy  program.  Currently  the  university  is  using  Courselnfo  for 
course  creation  , however  some  interest  has  been  expressed  regarding  WebCT,  another  on-line  course 
creation  program.  The  instructional  methods  were  similar  to  those  used  in  Task  II.  Continued  support  has 
been  provided  to  faculty  to  expand  usage  of  the  program. 
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Courselnfo  and  WebCT  Capabilities 


Some  of  the  major  features  of  these  programs  include: 

• General  display  of  textual  information,  like;  a syllabus,  instructors  vitas,  class  handouts,  and 
assignment  materials. 

• Bulletin  boards 

• Chat  rooms 

• Web  page  creators 

• E-mail 

• On-line  testing 

• Grouping  organizers 
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ICL7076-8076:  Seminar  in  Wbrkshop  Design 


Seminar  m Workshop  Design 


Announcements 

Posted  in  the  Last  T\so  Weeks 

AmAnnoun^^Mifromyour'Iwtnjctpr  Posted  by  on  Jan  18.  2000 

Welcome  to  Seminar  in  Workshop  Design.  The  primary  objective  for  this  course  is  to 
develop  skills  for  designing  and  delivering  a workshop  of  your  choice.  The  workshop  that 
you  design  will  be  ready  for  implementation  by  the  end  of  the  semester.  You  will  learn 
presentation  skills  along  with  the  process  of  developing  an  instructor's  manual  and  a 
student's  guide  for  your  workshop.  This  seminar,  in  workshop  design  emphasizes  the 
importance  of  adult  learning  in  the  context  of  "hands-on"  learning. 
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Figure  2.  Example  of  Courselnfo  Main  Screen 


Involvement  of  Faculty  in  the  Creation  of  On-line  Courses 
Current  Status 

Q 15%  of  faculty  are  currently  using  the  software  for  on-line  courses. 

G Initially  only  5%  expressed  any  interest  in  learning  an  on-line  course  program. 

□ An  additional  1 0%  have  express  interest  or  are  currently  being  instructed  on  the  program. 


133 


645 


□ By  the  end  of  the  second  semester  of  the  project,  we  plan  to  have  50%  of  the  faculty  using  the  on- 
line course  program. 

□ Small  and  Large  group  workshops  are  also  being  offered  at  the  university  on  creating  on-line 
courses. 

Conclusions  Regarding  On-line  courses 

The  faculty  have  found  on-line  courses  to  be  beneficial  for  the  following  reasons. 

• The  consistency  of  instructional  materials. 

• Reduction  in  duplicating  materials. 

• Increase  in  active  participation  of  students. 

• Promoting  self-reliance  among  the  students. 

• A new  dimensions  to  the  class  instruction  instead  of  straight  lecture. 

In  regards  to  the  instructional  method  used,  it  should  be  noted  that  the  faculty  found  the  one-on- 
one  instruction  enabled  them  to  learn  the  software  program  better  then  group  sessions.  However,  the  group 
sessions  allowed  them  to  express  their  concerns  regarding  the  broader  issues  of  on-line  instruction  among 
their  peers.  Regardless,  the  number  of  faculty  interested  in  trying  the  on-line  course  format  is  increasing 
each  semester. 

Conclusion 

Overall  the  experience  of  working  with  the  faculty  to  improve  their  use  of  technology  in  the 
classroom  has  been  positive.  Within  one  year  the  entire  department  has  established  a web  presence  and 
nearly  half  have  moved  onto  the  more  advanced  usage  of  technology  over  the  Internet.  As  more  of  the 
faculty  become  aware  of  the  benefits,  this  technology  can  bring  to  the  classroom,  the  greater  the  number 
will  want  to  utilize  it  in  their  instruction.  It  is  predicted  that  by  the  Summer  of  2001 , 80%  or  higher  of  the 
faculty  will  be  using  this  Internet  technology  in  their  classroom. 

1CL  Home  Page:  http://www.people.memphis.edu/~coe_icl/ 
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Abstract 

During  the  1997-8  academic  year,  66%  of  the  nation ' s 2 -year  and  4 -year  postsecondary  education 
institutions  offered  distance  education  courses,  and  another  20% planned  to  start  offering  such 
courses  (primarily  via  the  Internet)  by  2001.  Currently,  at  least  45%  of  these  institutions  offer  full 
college  courses  via  the  Internet.  Within  20 years,  on-line  classes  with  as  many  as  1 ,000  students 
will  replace  traditional  lecture  courses.  Distance  learning  is  poised  for  a 33%  annual  growth 
over  the  next  several  years.  The  number  of  college  students  enrolled  in  distance  -learning  courses 
will  reach  2.2  million  in  2002.  By  2002,  85%  of  two - and  four-year  colleges  will  offer  distance- 
learning courses.  On-line  course  design  and  facilitation  is  still  uncharted  territory  for  many 
faculty.  To  ensure  that  web-based  courses  are  adequately  developed  and  facilitated,  we  must 
carefully  critique  them.  This  paper  presents  a checklist  of 2 5 course  design  principals  faculty  can 
use  to  critique  or  evaluate  their  on-line  course  sites  prior  to  launch. 

Running  head:  On-line  Course  Design 

Web-based  (on-line)  courses  are  innovative  approaches  to  distance  education,  where  the  learning 
and  teaching  process  takes  place  via  the  technologies  and  methodologies  of  the  World  Wide  Web,  rather 
than  the  traditional  classroom.  Whether  around  the  comer  from  campus  or  even  around  the  world  from  it, 
an  increasing  number  of  college  students  are  registering  for  classes  that  no  longer  meet  in  a predetermined 
space  (e.g.,  a lecture  hall)  and  time.  These  classes  are  meeting  at  a “distance”  from  the  brick  and  mortar 
university  campus. 

Both  historically  and  currently,  the  terms  “distance  education”  or  “distance  learning”  are  applied 
interchangeably  by  many  different  researchers  and  practitioners  in  a great  variety  of  settings  and  global 
locations.  “Its  hallmarks  are  the  separation  of  teacher  and  learner  in  space  and/or  time,  the  discretionary 
control  of  learning  by  the  student  rather  than  the  distant  instructor,  and  noncontiguous  communication 
between  student  and  teacher,  mediated  by  print  or  some  form  of  technology  “ (Sherry,  1 996). 

According  to  a report  by  the  National  Center  For  Education  Statistics  (NCES)  (1999)  which 
collected  information  during  the  12-month  1997-98  academic  year,  66%  of  the  nation's  2 -year  and  4-year 
postsecondary  education  institutions  offered  distance  education  courses,  and  another  20%  of  the  institutions 
planned  to  start  offering  such  courses  within  the  next  three  years.  There  were  an  estimated  1,661,100 
enrollments  in  all  distance  education  courses,  and  1,363,670  enrollments  in  college-level,  credit- granting 
distance  education  courses,  with  most  of  these  at  the  undergraduate  level. 

For  the  purposes  of  the  NCES  study,  distance  education  refers  to  “education  or  training  courses 
delivered  to  remote  (off-campus)  location(s)  via  audio,  video  (live  or  prerecorded),  or  computer 
technologies,  including  both  synchronous  and  asynchronous  instruction”  (1999).  The  distance  education 
technologies  included  video  (two-way  video  with  two-way  audio,  one-way  live  video,  and  one-way 
prerecorded  video),  audio  (two-way  audio  transmission  and  one-way  audio  transmission),  and  Internet- 
based  technologies  (Internet  courses  using  synchronous  computer-based  instruction  such  as  interactive 
computer  conferencing,  and  Internet  courses  using  asynchronous  computer-based  instruction  such  as  e- 
mail,  listserves,  and  web-based  courses).  Of  the  institutions  offering  and/or  planing  to  offer  distance 
education  in  the  next  three  years,  the  technology  of  choice  was  Internet-based  technologies.  This  finding 
suggests  that  Internet  technologies  will  be  a growing  mode  of  delivery  among  postsecondary  institutions  . 

Another  indicator  of  the  growth  of  on-line  Internet  courses  comes  from  forecaster  William  A. 
Draves  of  the  Learning  Resources  Network,  a distance  learning  industry  group.  According  to  Draves, 
within  20  years,  on-line  classes  with  as  many  as  1,000  students  will  replace  traditional  lecture  courses  on 
campus  (Cameval,  2000).  He  also  believes  that  universities  should  slash  tuition  for  on-line  courses  to 
about  S 1 00  to  increase  access  to  those  in  lower  income  brackets.  He  does  not  believe  that  on-line  education 


5 Higher  education  institutions  are  a subset  of  postsecondary. 
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will  drive  traditional  education  out  of  business,  rather  it  will  replace  most  lecture-based  courses  on  campus. 
Traditional  classes  will  cater  mostly  to  small-group  discussions. 

Growth  is  also  predicted  by  the  International  Data  Corporations  (IDC).  “Distance  learning,  where 
student  and  teacher  are  connected  by  technology  rather  than  participating  in  a classroom,  is  becoming  a 
viable  option  to  traditional  teaching  methods,  and  is  poised  for  major  growth  [33%  annually]  over  the  next 
several  years”  (IDC,  2000).  According  to  IDC’s  research,  the  number  of  college  students  enrolled  in 
distance-learning  courses  will  reach  2.2  million  in  2002,  up  from  710,000  in  1998.  The  number  of  higher 
education  institutions  offering  distance-learning  programs  is  also  growing  dramatically.  According  to  the 
IDC,  by  2002,  85%  of  two-year  colleges  will  be  offering  distance-learning  courses,  up  from  58  percent  in 
1 998.  Eighty-four  percent  of  four-year  colleges  will  be  offering  distance-learning  courses  in  2002,  up  from 
62%  in  1998. 

Despite  the  established  base  of  on-line  courses,  on-line  course  design  and  facilitation  is  still 
uncharted  territory  for  many  college  and  university  faculty.  Many  faculty  members  struggle  with  how  to 
successfully  use  the  available  tools  and  technologies  to  organize  instructional  content  into  well-crafted 
teaching  systems  that  support  learning.  Integrating  technology  into  instruction  remains  the  single  most 
important  information  technology  challenge  confronting  American  colleges  and  universities  over  the  next 
two  to  three  years,  according  to  data  from  the  Campus  Computing  Project  (1999),  a national  survey  of 
information  technology  in  US  higher  education.  Thirty-nine  percent  of  the  institutions  participating  in  the 
survey  identify  instructional  integration  as  their  single  most  significant  instructional  technology  (IT) 
challenge,  up  from  33.2%  in  1998  and  29.6%  in  1997.  While  its  been  nearly  a quarter  of  a century  since 
the  appearance  of  the  first  desktop  computers  on  college  campuses,  the  major  challenges  involve  human 
factors  - assisting  students  and  faculty  to  make  effective  use  of  new  technologies  in  ways  that  support 
teaching,  learning,  instruction  and  scholarship.  Fortunately,  most  campuses  have  IT  development  programs 
(74.5%)  and  campus  support  centers  (66.6%)  to  assist  faculty  in  bringing  technology  resources  into  their 
courses.  However,  the  report  does  not  indicate  the  extent  to  which  faculty  actually  avail  themselves  of  this 
assistance  and  support. 

In  spite  of  the  apparent  learning  curve,  higher  education’s  interest  in  using  the  Internet  to  provide 
instruction  and  courses  steadily  increases.  According  to  the  Campus  Computing  Project  survey  (1999), 
since  1 998,  the  percentage  of  all  college  courses  making  use  of  electronic  mail  or  Web  resources  increased 
5%  to  53.4%  and  38.6%  respectively.  Twenty-eight  percent  of  college  faculty  utilizes  a course  web  page. 
Forty- five  percent  of  the  participants  in  the  1999  survey  report  that  their  institution  currently  offers  one  or 
more  full  college  courses  online  via  the  Internet  and  the  World  Wide  Web. 

Clearly,  distance  education  technologies  — and  specifically  on-line  college  courses  — are 
expanding  at  an  extremely  rapid  rate.  Far  too  often,  college  faculty  become  enamored  with  these 
technologies,  and  are  quick  to  make  available  web-based  courses  (or  web-enhanced  traditional  courses). 
They  do  so  without  adequately  addressing  the  design,  infrastructure  or  substance  of  their  course  sites,  much 
less  other  underlying  issues  such  as  the  new  roles  of  teacher,  site  facilitator,  and  student  in  the  distance 
learning  process.  Perhaps  this  is  due  to  the  proliferation  of  tools  (e.g.,  FrontPage,  Blackboard/Courselnfo) 
which  dramatically  decrease  the  time  and  effort  required  to  create  and  publish  web  documents. 
Unfortunately,  quick  development  time  does  not  guarantee  a well-crafted  course  site. 

Course  Design  Principals 

To  ensure  that  web-based  courses  are  adequately  developed  and  facilitated  in  a manner  that  will 
secure  and  maintain  student  interest,  motivation,  satisfaction  and  success,  we  must  carefully  critique  them 
prior  to  actually  going  "live".  This  paper  will  present  a checklist  of  factors  faculty  can  use  to  critique  or 
evaluate  their  on-line  course  sites  prior  to  launch. 

Authority 

Look  at  your  web  resources  (e.g.,  links  to  other  web  sites)  with  a critical  eye.  What  is  the  basis  of 
the  authors’  authority?  That  is,  how  have  they  established  themselves  as  someone  with  legitimate  expertise 
in  whatever  area  their  website  is  about?  You  should  first  consider  if  the  author  is  a well-regarded  name  that 
you  recognize.  If  not,  then  you  should  check  to  see  if  the  on-line  document  has  some  kind  of  bibliography, 
that  is,  a list  of  references  used  to  develop  the  information.  Next,  look  for  the  author’s  biography,  if  there 
is  one.  Is  the  author  affiliated  with  an  educational  institution,  research  laboratory,  governmental  agency,  or 
other  reputable  organization  related  to  the  topic  of  the  document?  Finally,  consider  e-mailing  the  author 
and  tactfully  ask  them  how  they  came  to  be  an  “authority”  on  the  topic. 
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Bias 

Bias  applies  to  two  situations  here.  First,  make  sure  your  text  and  graphics  are  not  biased  or 
insensitive.  Are  there  indications  of  gender  or  racial  biases  and  stereotyping  in  text  or  graphics?  You  also 
want  to  check  for  bias  in  your  included  web  resources.  Information  is  biased  when  it  provides  only  one 
opinion  or  point  of  view.  It  is  not  uncommon  for  a web  search  on  a particular  term  (such  as  multimedia 
development)  to  result  in  hundreds  of  “hits”  consisting  primarily  of  commercial  sites  or  other  for-profit 
entities.  Rather  than  providing  unbiased  information  - what  you  expect  from,  say,  an  encyclopedia  - they 
provide  words  that  try  to  persuade  (e.g.,  advice)  or  sell.  Check  to  see  what  organization  is  sponsoring  the 
page,  and  follow  the  link  to  that  organization  if  there  is  one.  Finally,  check  to  see  if  the  page  is  actually  an 
advertisement  or  opinion  thinly  disguised  as  a scholarly  source  of  information.  Many  commercial  sites 
include  “white  papers”  or  case  studies  that  appear  to  be  unbiased  research  studies,  but  only  serve  to 
increase  sales  of  their  product. 

Citations 

Have  you  included  full  citations,  and  in  the  proper  format?  If  you  post  a document,  such  as  a 
scanned  journal  article,  be  sure  to  include  a full  citation.  See  http://www.apa.org/ioumals/webref.html  for 
the  APA’s  recommended  electronic  reference  formats. 

Dates 

Every  credible  web  site  includes  the  date  it  was  created  and  the  date  of  the  last  update.  Also 
consider  the  date  (age)  of  your  web- resources  and  other  posted  information.  Naturally,  it  should  be  current, 
however,  you  should  avoid  discarding  seminal  works  just  because  they  have  an  “older”  publication  date. 
Error  messages 

If  an  error  occurs,  learners  should  be  informed  what  the  error  is,  why  it  occurred  and  what  they 
can  do  to  fix  it.  Communicate  in  the  user’s  vocabulary.  It  is  a good  idea  to  utilize  server  software  that 
allows  you  to  customize  the  error  messages.  For  example,  if  an  on-line  learner  at  Okanagan  University 
College  (http://www.onlinecourses.ouc.bc.ca/)  mistypes  his/her  log-on  ID,  the  error  messages  reads: 

“This  server  could  not  verify  that  you  are  authorized  to  access.  Typical  causes 
of  this  are  you  misspelled  your  LoginID  or  password,  you  used  spaces  or  punctuation  in 
your  LoginID  or  Password,  you  entered  your  LoginID  or  Password  in  the  wrong  case 
(upper  or  lower)  or  in  mixed  case,  you  clicked  on  the  wrong  course  or  the  wrong  section 
of  your  course.  Return  to  the  Course  List  and  try  again.” 

Frames 

Frames  can  be  great  for  separating  navigation  components  from  content.  Are  your  frames  neatly 
arranged,  or  do  they  muck  up  the  screen,  potentially  confusing  the  learner?  Depending  on  which  course 
management  system  you  utilize,  such  as  Blackboard/Courselnfo,  you  may  have  no  choice  about  using 
frames,  and  likewise,  an  inability  to  provide  a no-frames  version  of  your  site. 

Graphics 

Have  you  chosen  the  proper  format  for  your  graphics?  The  best  rule  of  thumb  is  to  use  JPEG  for 
pictures  (i.e.,  photos)  and  GIF  for  graphics,  charts,  clip  art,  etc.  Also,  do  not  save  your  images  with  more 
than  256  colors  and  75  DPI.  Set  graphics  to  interlacing  such  that  they  become  more  focused  as  they  load. 
This  gives  the  learner  something  to  watch  and  is  less  likely  to  become  impatient  with  a slow  download. 
Finally,  provide  an  alternative  text  that  appears  while  an  image  is  loading,  when  the  ‘show  images’  option 
on  the  browser  is  turned  off,  and  in  text-only  browsers. 

Help 

Is  "help"  available  for  those  requiring  it?  While  we  strive  to  create  web  sites  that  are  intuitive  and 
user  friendly,  we  should  still  provide  assistance  for  those  whom  require  it.  Include  access  to  help  on  how  to 
use  the  site.  The  help  should  explain  what  all  the  features,  buttons,  and  sections  of  the  site  are.  Frequently 
asked  questions  (FAQs)  are  valuable  as  well.  If  you  have  a help  section  or  a FAQ,  consider  how  helpful  it 
really  is. 

Icons 

An  icon  is  a symbol  or  graphical  representation  of  a concept,  such  as  an  image  of  a house 
representing  the  home  page.  Do  the  icons  clearly  represent  what  is  intended  to  others  beside  you?  Make 
sure  your  icons  are  consistent,  well  defined,  and  comprehendible  (across  cultures). 

Join 

Have  you  provided  information  on  how  interested  students  can  join  your  class?  Provide  specific 
detail  on  the  enrollment  and  registration  process.  Occasionally,  web  surfers  will  locate  your  class  and  want 
to  join  it;  even  if  they  are  not  enrolled  at  your  university. 
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Keeping  records 

Does  your  site  keep  records  of  student  performance?  Whether  records  are  kept  in  a student 
accessible  online  grade-book,  or  quiz  scores  are  automatically  e-mailed  to  you,  check  the  functionality  of 
these  features. 

Links 

Check  your  site  for  links  that  are  outdated,  gone,  or  simply  inaccurate.  Are  all  the  links  clearly 
visible,  accurately  labeled  and  understandable?  You  should  check  your  site  at  least  once  per  semester  for 
unreliable  links,  or  what  is  called  ‘link-rot’.  A reliable  link,  on  the  other  hand,  is  up-to-date,  active  and 
does  not  reference  a site  that  has  moved  (with  no  indication  of  the  new  URL).  Be  sure  to  inform  learners 
when  clicking  on  a link  will  ‘surf  them  off  the  course  site  to  a separate  website,  rather  than  to  another  page 
on  the  course  site.  Links  to  documents  and  files  should  be  annotated,  informing  learners  of  the  type  of  file 
or  information  they  are  linking  to  (e.g.,  video,  sound,  text,  etc.).  Finally,  check  all  links  on  the  site  for 
relevance  and  appropriateness. 

Multimedia 

Does  the  use  of  animation,  video  or  audio  clips  contribute  to  the  understanding  of  the  information? 
Given  bandwidth  constraints  and  limitations,  make  sure  any  media  you  embed  is  really  necessary.  Music 
playing  in  the  background  of  the  home  page  is  probably  a ‘bell  & whistles’  you  can  do  without.  Is  file  size 
information  included  for  all  downloadable  media  clips  (e.g.,  combustion.avi  [947kD?  Finally,  if  you 
include  streaming  media  on  your  site,  be  sure  to  either  offer  learners  the  means  to  choose  connection  speed 
(e.g.,  28. 8K,  56K,  Tl),  or  stream  your  media  at  a low  data  rate.  This  will  enable  those  with  low  connection 
speeds  to  view  your  content. 

Navigation 

Learners  should  always  be  informed  where  they  are  within  the  hierarchy  of  pages  on  the  course 
site,  and  how  to  get  to  their  next  destination.  Check  that  all  of  the  locations  (e.g.,  pages)  within  the  site  are 
labeled.  Have  you  provided  clear,  simple  ways  to  get  to  other  parts  of  your  site?  Learners  will  find 
navigation  links,  site  maps,  search  mechanisms,  indices  and  table  of  contents  quite  useful.  Your  first  task  is 
to  provide  navigation  control  on  every  page.  Don't  rely  solely  on  the  navigation  controls  in  the  browser. 
Does  each  page  provide  a way  to  return  to  the  home  page?  Check  the  number  of  steps  required  for 
navigating  from  one  place  to  another.  It  should  be  kept  to  a minimum.  Check  the  length  of  your  pages.  If 
scrolling  is  necessary  to  read  a single  page  (as  you  might  expect  if  you  placed  an  entire  500-line  syllabus  on 
one  page),  provide  links  within  the  long  pages  by  using  targets.  Try  to  keep  your  navigation  hierarchy  as 
flat  as  practical.  There  should  be  no  more  than  five  levels  and  preferably  fewer  than  three.  Check  for 
dead-ends,  which  are  links  to  destinations  that  provide  no  means  to  return  back.  Finally,  be  sure  to  indicate 
the  distance  and  placement  of  the  learner  within  instructional  sequences  (e.g.,  page  5 of  1 5,  page  6 of  1 5, 
etc.). 

Organization 

Are  your  online  documents  and  other  information  well  organized?  Course  management  systems 
such  as  Blackboard/Courselnfo  impose  some  organization  upon  you,  with  an  area  for  announcements,  an 
area  for  assignments,  etc.  The  goal  is  to  keep  related  information  grouped  in  an  orderly  fashion.  For 
example,  one  natural  approach  is  to  place  different  information  (e.g.,  office  hours,  reading  list)  on  separate 
pages,  and  to  use  proper  headings  within  pages. 

Printing 

Learners  may  want  to  print  all  or  some  of  your  web  pages.  Some  information  is  appropriate  for 
printing  and  some  is  not.  For  example,  a page  designed  to  present  streaming  audio  or  video  cannot  be 
printed  in  a meaningful  fashion  because  the  data  are  not  amenable  to  the  print  medium.  On  the  other  hand, 
text  that  users  may  want  to  keep  as  reference  material  is  appropriate  for  printing.  If  students  will  want  to 
print  the  web  pages,  which  is  highly  likely,  design  them  such  that  they  format  well  in  printed  and  on-screen 
form.  To  do  this,  be  mindful  of  page  width  and  length.  To  facilitate  printing  multiple  pages  as  a collection, 
you  should  provide  a means  for  them  to  do  so.  Simply  supply  a link  to  one  long  page  that  is  comprised  of 
all  the  text  from  the  course  site.  This  way,  the  print  command  can  be  issued  just  once. 

Quick  & dirty 

“Quick  & dirty”  means  cheaply  made  or  done,  or  of  inferior  quality.  Does  your  site  look  quick  & 
dirty?  Does  it  appear  as  if  you  created  it  one  hour  before  the  arrival  of  your  first  online  student?  Carefully 
check  your  site  for  misspellings,  improper  diction  and  syntax,  missing  data  (e.g.,  graphics),  etc. 
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Required  plug-ins 

If  Java,  ActiveX  extensions,  or  plug-ins  are  employed,  do  they  actually  improve  the  site?  Make 
sure  required  plug-ins  or  other  helper  applications  are  clearly  identified,  preferably  right  on  the  homepage. 

Structure 

Stracture  refers  to  the  arrangement  and  organization  of  the  elements  of  the  course  site,  and  the 
relationship  of  those  parts  to  each  other.  In  other  words,  how  well  is  your  site  constructed?  Do  your  web 
pages  follow  good  graphic  and  message  design  principles?  Do  the  graphics  serve  a function,  or  are  they 
merely  decorative?  Are  backgrounds  or  other  visual  elements  distracting  or  cluttered?  If  there  an  element 
of  creativity  does  it  add  to  or  detract  from  the  document  itself?  Does  the  visual  design  enhance  usability 
and  understanding,  or  is  it  distracting?  Make  sure  that  essential  instructions  appear  before  interactive 
portions.  Assure  that  all  the  parts  of  the  site,  such  as  a Flash  animations,  work.  Is  the  site  intuitive,  or  are 
parts  likely  to  be  misunderstood?  Check  the  structure  of  the  site  for  stability.  Features  should  not 
disappear  and  reappear  between  visits.  Any  interactions  that  involve  private  information  should  be  secure. 
How  well  do  the  features  of  the  site  work  for  learners  with  older  browsers?  The  site  design  should  be 
appealing  to  and  appropriate  for  the  intended  audience.  How  long  does  it  take  for  your  pages  to  load?  The 
pages  should  consistently  load  without  problems;  stability  is  important.  Overall,  your  goal  is  for  the  design 
elements  and  features  — such  as  searchable  databases,  animations,  graphics,  sound  files,  and  transitional 
pages  - to  enhance  and  not  hinder  the  accessibility  and  enjoyment  of  the  site. 

Text 

Does  the  text  follow  basic  rules  of  grammar,  spelling,  punctuation  and  literary  composition?  Is 
attention  paid  to  the  needs  of  the  disabled  who  often  require  a larger  font  to  read  the  materials?  Is  the  text 
concise,  or  does  it  ramble  on,  resulting  in  excessive  scrolling  (which  should  be  avoided  anyway)?  The  text 
should  be  easy  to  read,  and  not  cluttered  by  distracting  graphics,  fonts,  and  backgrounds.  Check  the  text 
for  sufficient  contrast,  and  adequate  print  size.  Is  the  text,  such  as  headings,  clear  and  descriptive,  or  does 
it  use  jargon  meaningful  only  to  you? 

User  friendly 

Does  the  site  look  and  feel  user  friendly?  Is  it  easy  to  find  your  way  around  and  locate  a particular 
page  from  any  other  page?  Can  the  learner  interact  in  satisfying  ways?  The  information  on  the  site  should 
be  easy  to  find  and  easy  to  use.  Users  should  be  able  to  maneuver  around  the  site  easily.  All  interactive 
features  should  be  explained  clearly. 

Virus -free 

Does  your  course  require  students  to  download  files  from  the  site,  such  as  executable  programs  or 
Word  documents?  If  so,  virus  scan  them  to  assure  they  are  clean.  (Authors  note:  I regularly  receive 
uploaded  assignments  from  on-line  students  that  have  viruses,  especially  Word  documents). 

Who  are  you? 

Your  site  should  indicate  whom  you  are  (your  name),  where  you  work  (the  institution  with  which 
you  are  affiliated),  what  your  credentials  are  (position/title),  and  how  your  students  can  get  in  touch  with 
you  (e-mail  address,  phone  number,  and  mailing  address).  If  other  individuals,  groups,  or  organizations 
provided  assistance  in  the  creation  of  your  course  site,  make  sure  you  give  credit  where  credit  is  due. 
Finally,  if  the  development  of  the  site  was  funded  or  otherwise  supported  by  an  individual,  group,  or 
organization  other  than  you,  again,  say  so. 

Xerox 

Make  sure  you  have  a back-up  (a  Xerox  copy  and  disk  copy)  of  your  web  pages  and  other  on-line 
material  and  documents.  In  the  event  lightning  strikes  your  server,  do  you  have  a back-up  of  your  files,  or 
will  you  be  ‘up  the  creek  without  a paddle’? 

Yo-yo  effect 

When  viewing  an  instructional  sequence,  such  as  steps  in  a process,  do  learners  have  to  repeatedly 
return  to  a higher-level  page  before  examining  the  next  item  in  what  seems  like  a logical  sequence  to  them? 
Consider  a lesson  on  the  3 to  5 steps  to  change  a light  bulb.  Rather  than  incorporating  a logical 
navigational  structure  where  the  learners  can  click  on  back  or  forward  arrows  to  navigate  through  the 
lesson,  the  yo-yo  effect  forces  them  to  first  return  to  the  main  menu  in  order  to  select  the  next  task.  This 
‘up -and- down -up- an  d-d  own’  should  be  avoided. 

Zero 

Now  that  you’ve  reached  the  end,  you  should  have  zero  faults  with  your  course  site! 
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Abstract 

This  study  investigated  the  effects  of  audio,  animation  and  spatial  ability  in  a computer  based 
instructional  program  for  biology1.  The  program  presented  instructional  material  via  text  or  audio 
with  lean  text  and  included  eight  instructional  sequences  presented  either  via  static  illustrationsor 
animations.  High  school  students  enrolled  in  a biology > course  were  blocked  by  spatial  ability  and 
randomly  assigned  to  one  of  four  treatments  (Text-Static  Illustration,  A udio-Static  Illustration, 
Text-Animation,  A udio-Animation).  The  study  examined  the  effects  of  instructional  mode  (T ext  vs. 
Audio),  illustration  mode  (Static  Illustration  vs.  Animation)  and  spatial  ability  (Low  vs.  High)  on 
practice  and  posttest  achievement,  attitude  and  time. 

Results  for  practice  achievement  indicated  that  high  spatial  a bility  participants  achieved  more 
than  low  spatial  ability  participants.  Similar  results  for  posttest  achievement  and  spatial  ability 
were  not  found.  Participants  in  the  Static  Illustration  treatments  achieved  the  same  as  participants 
in  the  Animation  treatments  on  both  the  practice  and  posttest.  Likewise,  participants  in  the  Text 
treatments  achieved  the  same  as  participants  in  the  Audio  treatments  on  both  the  practice  and 
posttest. 

Findings  for  time-in-program  and  time-in-instruction  indicated  that  participants  in  the  Animation 
treatments  took  significantly  more  time  than  participants  in  the  Static  Illustration  treatments.  No 
time  differences  of  any  type  were  found  for  participants  in  the  Text  versus  Audio  treatments. 
Implications  for  the  design  of  multimedia  instruction  and  topics  for  future  research  are  included. 

Introduction 

Multimedia  computer-based  instruction  (CBI)  is  increasingly  used  as  an  adjunct  to  traditional 
instruction  in  schools  and  corporations.  CBI  has  typically  incorporated  text  and  graphics,  but  technology 
now  exists  which  allows  the  easy  and  inexpensive  inclusion  of  audio  and  animation  into  CBI  programs. 
However,  little  research  exists  to  support  the  notion  that  adding  audio  or  animation  to  CBI  improves 
learning. 

Research  in  audio-text  instruction  is  contradictory  with  some  studies  indicating  that  text  plus  audio 
is  more  effective  than  either  alone  (Enerson  & Turney,  1984;  Hartman,  1961;Lauret,  1998;Menne& 
Menne,  1 972;  Nasser  & McEwen,  1 976).  Other  studies  indicate  that  text  plus  audi o is  not  more  effective 
than  either  alone  (Barron  & Atkins,  1994;  Barron  & Kysilka,  1993;  Barton  & Dwyer,  1987;  Fumham, 
Gunter  & Green,  1990;  Nugent,  1982;  Rehaag  & Szabo,  1995;  Van  Mondfrans  & Travers,  1964).  These 
contradictory  results  can  be  explained  by  Paivio’s  dual  coding  theory  which  proposed  that  two  separate 
systems  are  involved  in  cognition,  one  for  verbal  information  and  another  for  image  formation  (Paivio, 
1986).  In  Paivio’s  view,  spoken  and  written  language  are  both  verbal  information  and  are  encoded  into 
verbal  representations  (Clark  & Paivio,  1991).  In  terms  of  dual  coding  theory,  redundant  audio  is  single 
channel  verbal  information  and  would  not  be  expected  to  increase  learning. 

Audio  combined  with  animation  is  a relatively  new  research  field  that  evolved  from  research  into 
the  effective  integration  of  text  with  illustrations.  Some  audio-animation  research  has  found  the 
combination  an  effective  technique  (Lee,  1 996;  Mayer  & Anderson,  1991;  Mayer  & Anderson,  1 992; 
Mayer  & Moreno,  1 998;  Mayer  & Sims,  1 994;  Moreno  & Mayer,  1 999),  while  other  such  research  has  not 
found  positive  results  (Childress,  1995;  Lee,  1997;  Palmiter  & Elkerton,  1993;  Wilson,  1998). 

The  role  spatial  ability  plays  in  learners’  interpretation  and  comprehension  of  animated  and  static 
graphics  is  unclear.  Some  researchers  have  found  animation  beneficial  to  low  spatial  ability  learners 
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(Blake,  1977;  Hays,  1996).  Other  studies  have  found  animation  more  beneficial  to  high  spatial  ability 
learners  (Hegarty  & Sims,  1994;  Hegarty  & Steinhoff,  1997;  Mayer  & Sims,  1994). 

The  present  study  investigated  the  effects  of  audio,  animation  and  spatial  ability  utilizing  a 
multimedia  CBI  program  concerning  a scientific  process.  The  major  independent  variables  were 
instructional  mode  (text  versus  audio),  illustration  mode  (static  versus  animated)  and  spatial  ability  (low 
versus  high). 

Instructional  mode  consisted  of  two  versions,  text  and  audio.  In  the  text  version,  the  instruction 
was  presented  as  screen  text,  while  in  the  audio  version,  the  i nstruction  was  presented  as  spoken  words  with 
limited  screen  text.  The  spoken  words  of  the  audio  version  matched  the  text  of  the  text  version. 

There  were  two  versions  of  illustration  mode,  static  and  animated.  The  static  version  consisted  of  a 
graphic  depicting  the  process  with  no  visual  movement  to  show  the  process  in  operation,  while  the 
animated  version  showed  the  process  with  visual  movement  to  demonstrate  the  process  in  operation. 

Spatial  ability  represented  another  variable  in  this  study.  A ll  participants  were  classified  as  low  or 
high  spatial  ability  based  on  learners'  scores  on  the  Paper  Folding  Test  (Ekstrom,  French,  & Harmon, 
1976). 

Research  Questions 


What  is  the  effect  of  instructional  mode,  illustration  mode  and  spatial  ability  on  achievement,  amount  of 
invested  mental  effort  and  time? 

Does  instructional  mode,  illustration  mode  and  spatial  ability  interact  to  influence  achievement,  amount  of 
invested  mental  effort  and  time? 

Method 

Participants 

One  hundred  and  nine  students  from  an  urban  high  school  biology  course  participated  in  this 
study.  Participants  were  blocked  by  spatial  ability  and  randomly  assigned  to  one  of  four  treatments  (Text- 
Static  Illustration,  Audio-Static  Illustration,  Text-Animation,  Audio- Animation). 

Materials 

A CBI  program,  The  Cell  Cycle , was  the  primary  instructional  material.  This  CBI  covered  mitosis 
and  meiosis  and  took  40-70  minutes  to  complete.  The  CBI  was  based  upon  the  objectives  and  content  of  the 
biology  course  and  included  information,  examples,  activities,  practice  with  feedback  and  review.  Figure  1 
shows  sample  instructional  screens  for  Text-Static  Illustration  and  Audio-Static  Illustration  program 
versions. 

Procedures 

A spatial  ability  test  was  administered  to  the  participants  approximately  one  week  prior  to  the 
study.  Scores  from  all  participants  were  ranked  and  median  split  to  classify  participants  as  high  or  low 
spatial  ability.  Participant  assignment  to  each  of  the  four  treatments  was  counterbalanced  by  spatial  ability. 
On  the  first  day  of  the  study,  participants  received  instructions  from  the  researcher  and  worked  through  the 
CBI  program.  On  the  second  day  of  the  study,  participants  completed  the  CBI  program,  an  amount  of 
invested  mental  effort  survey  and  a posttest.  All  events  occurred  during  normally  scheduled  class  time. 

Criterion  Measures 

There  were  three  criterion  measures  employed  in  this  study:  an  amount  of  invested  mental  effort 
survey,  practice  item  results  and  posttest  scores.  En-route  time  data  was  also  examined. 

A 3-item  Likert  scaled  (5  point  scale  from  Strongly  Agree  to  Strongly  Disagree)  amount  of 
invested  mental  effort  survey  was  administered  prior  to  the  posttest.  This  survey  had  a reliability  of  0.57. 
The  three  items  were  similar  to  those  developed  by  Salomon  ( 1 984)  and  concerned  the  amount  of  effort  and 
concentration  expended  by  the  participants  as  well  as  how  well  they  thought  they  understood  the  material. 
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Achievement  was  measured  by  a 27-item  posttest.  The  posttest  included  1 5 selected  response  and 
12-constructed  response  items  with  each  item  worth  one  to  three  points  for  a total  of  30  possible  points. 
The  reliability  of  the  posttest  was  0.82. 

The  practice  items  were  similar  in  form  and  content  to  the  posttest  and  included  17  selected 
response  and  1 1 -constructed  response  items  with  each  item  worth  one  to  three  points  for  a total  of  30 
possible  points.  The  reliability  of  the  practice  items  was  0.70. 

Results 


No  differences  in  posttest  achievement  for  instructional  mode,  illustration  mode  or  spatial  ability 
were  found.  The  overall  mean  for  the  posttest  was  17.34  (57.8%).  Practice  achievement  differences  by 
spatial  ability  were  found  (see  Table  1 ).  High  spatial  ability  participants  had  greater  achievement  on 
practice  items  than  low  spatial  ability  participants.  The  overall  mean  practice  score  was  1 5.45  (5 1 .5%). 

There  was  a significant  difference  between  High  and  Low  spatial  ability  participants  for  amount  of 
invested  mental  effort  (see  Table  2).  Low  spatial  ability  participants  indicated  a greater  amount  of  invested 
mental  effort  than  high  spatial  ability  participants.  The  overall  mean  for  invested  mental  effort  was  4.04. 

Three  types  of  en  route  time  data  were  collected.  For  the  purposes  of  the  study,  total  time-in- 
program was  defined  as  the  time  elapsed  between  the  participant  entering  and  exiting  the  CBI  program; 
time-in-practice  was  defined  as  the  time  the  participants  spent  completing  the  practice  items  within  the  CBI 
program;  and  time-in-instruction  was  defined  as  the  difference  between  total  time -in-program  and  time-in- 
practice.  Since  the  practice  screens  were  identical  in  all  four  treatments,  time-in-instruction  represented  the 
time  participants  spent  within  the  treatments  (Text  versus  Audio,  Static  illustration  versus  Animation). 
Time  data  revealed  that  participants  spent  significantly  more  time -in-program  and  time-in-instruction  for 
illustration  mode  with  animation  taking  longer  than  static  illustration  mode  (see  Tables  3 and  4). 


Table  L Practice  Achievement  Means  and  Standard  Deviations. 


Instructional  Mode 

Illustration  Mode 

Spatial  Ability 

Text 

M 75.95 

SD  4.47 

N 44 

Static 

M 15.59 

SD  4.68 

N 41 

Low 

M 14.41 
SD  4.54 
N 41 

Audio 

M 14.90 

SD  4.33 

N 40 

Animation 

M 15.33 

SD  4.19 

N 43 

High 

M 16.44 
SD  4.09 
N 43 

Table  2.  Amount  of  Invested  Mental  Effort  Means  and  Standard  Deviations. 


Instructional  Mode 

Illustration  Mode 

Spatial  Ability 

Text 

M 3.96 

SD  0.66 

N 51 

Static 

M 3.98 

SD  0.71 

N 56 

Low 

M 4.21 

SD  0.58 
N 50 

Audio 

M 4.11 

SD  0.63 

N 52 

Animation 

M 4.11 

SD  0.56 

N 47 

High 

M 3.87 

SD  0.67 
N 53 
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Mitosis 

fefUtoafe  fc?  the  prpc*m  of  eed 
replication  that  is  necessary  for  an 
organism  to  grow  or  repair  damage. 

During  mitosis,  the  ceil  nucleus 
divides  Into  two  identical  nuclei!. 
After  mitosis,  tfoa  ceO  cytoplasm 
divides  to  form  two  caUs-each 
genetically  Identical  to  the  original 
cell  Remember,  mitosis  results  in 
two  exact  duplicates  of  the  original 

CO0. 


Text-Static  Illustration  Version  Instructional  Screen 


j Mitosis 

• Necessary  for  growth  or  repair 

♦ Nucleus  divides  into  two 

* Cytoplasm  divides  into  two 

| * Results  In  two  exact  duplicates 


\<  <SS5> 


Corresponding  Audio-Static  Illustration  Version  Instructional  Screen 

Figure  L Sample  Instructional  Screens  for  Text-Static  Illustration  and  Audio-Static 

Illustration  program  versions . 
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Table  3.  Time -in -Pro gram  Means  and  Standard  Deviations. 


Instructional  Mode 

Illustration  Mode 

Spatial  Ability 

Text 

M 54.46 

SD  1 0.42 

N 44 

Static 

M 50.40 

SD  11.41 

N 41 

Low 

M 53.32 
SD  13.05 
N 37 

Audio 

M 51.89 

SD  11.38 

N 36 

Animation 

M 56.35 

SD  9.48 

N 39 

High 

M 53.29 
SD  8.73 

N 43 

Note.  Time-in-Program  is  reported  in  minutes. 


Table  4.  Time-in-lnstruction  Means  and  Standard  Deviations. 


Instructional  Mode 

Illustration  Mode 

Spatial  Ability 

Text 

M 42.51 

SD  7.67 

N 44 

Static 

M 39.16 

SD  8.89 

N 41 

Low 

M 42.17 
SD  10.25 
N 37 

Audio 

M 40.63 

SD  9.32 

N 36 

Animation 

M 44.30 

SD  7.17 

N 39 

High 

M 41.23 
SD  6.62 

N 43 

Note.  Time-in-Program  is  reported  in  minutes. 


Discussion 


Results  for  practice  achievement  indicated  that  high  spatial  ability  participants  achieved  more  than 
low  spatial  ability  participants.  These  findings  may  best  be  understood  in  terms  of  dual  coding  theory 
which  proposes  two  separate  systems  for  cognition — verbal  and  visual  (Paivio,  1986).  Under  dual  coding, 
images  such  as  static  illustrations  or  animations  are  organized  into  visual  representations  while  verbal 
information  such  as  text  or  audio  is  organized  into  verbal  representations.  Referential  connections  are  then 
built  between  the  verbal  and  visual  representations.  It  is  thus  likely  that  high  spatial  ability  participants  built 
visual  representations  more  easily  and  completely  than  did  low  spatial  ability  participants.  This  allowed 
high  spatial  ability  participants  to  invest  more  cognitive  resources  towards  integrating  verbal  with  visual 
information  and  resulted  in  higher  practice  achievement. 

While  results  for  practice  achievement  indicated  a relationship  with  spatial  ability,  similar  results 
for  posttest  achievement  were  not  found.  Both  high  and  low  spatial  ability  participants  achieved  more  on 
the  posttest  than  on  the  practice  and  high  spatial  ability  participants  achieved  more  than  low  spatial  ability 
participants.  However,  the  achievement  difference  between  low  and  high  spatial  ability  participants  was  not 
statistically  significant. 

One  potential  reason  for  this  lack  of  differences  in  posttest  achievement  by  spatial  ability  may  be 
related  to  the  design  of  the  CBI  program  used  in  this  study.  It  was  designed  to  be  instructionally  effective 
and  included  instruction,  interactive  activities,  practice  with  feedback  and  reviews.  Previous  studies  that 
have  investigated  spatial  ability  and  animation  or  animation  with  audio  have  found  significant  posttest 
achievement  differences  by  spatial  ability.  However,  these  studies  have  not  incorporated  practice  into  their 
designs  (Hays,  1 996;  Hegarty  & Sims,  1994;  Hegarty  & Steinhoff,  1997;  Mayer  & Sims,  1994).  Since  the 
current  study  did  include  practice,  it  might  be  more  appropriate  to  compare  practice  results  of  the  current 
study  to  the  posttest  results  of  the  previous  studies.  Viewed  thusly,  the  difference  in  practice  achievement 
found  for  the  current  study  supports  the  findings  for  posttest  achievement  differences  found  by  previous 
studies.  It  is  also  possible  that  the  effect  of  practice,  followed  as  it  was  in  the  CBI  program  by  a review, 
helped  low  spatial  ability  learners  compensate  for  their  lower  visualization  capabilities,  thus  raising  their 
scores  close  to  those  of  high  spatial  ability  learners. 
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Although  the  CBI  program  included  elements  of  effective  instruction,  overall  performance  on  both 
the  practice  and  posttest  was  quite  low  (5 1 .5%  for  the  practice,  57.8%  for  the  posttest).  This  was  probably 
due  to  several  factors.  As  with  most  science  topics,  the  CBI  program  was  heavily  laden  with  the  technical 
jargon  necessary  to  understand  the  content.  Considering  the  short  time  of  the  treatment,  participants  may 
have  been  unable  to  acquire  the  extensive  vocabulary  required  by  the  material  despite  activities 
incorporated  into  the  CBI  to  aid  in  learning  and  retaining  this  terminology.  In  addition,  this  course  was  the 
first  high  school  science  course  for  many  of  the  participants  and  they  may  have  had  few  existing  science 
concepts  with  which  they  could  relate  new  concepts.  To  be  concise,  the  CBI  covered  a lot  of  material  in  a 
relatively  short  amount  of  time.  It  is  possible,  and  indeed  the  overall  achievement  scores  suggest,  that  more 
time  and  practice  was  necessary  for  participants  to  master  the  material. 

Examination  of  the  Amount  of  Invested  Mental  Effort  (AIME)  results  indicated  significant 
differences  by  spatial  ability.  Low  spatial  ability  participants  reported  greater  AIME  than  high  spatial 
ability  participants.  This  result  can  be  explained  by  dual  coding  theory  (Paivio,  1 986).  Low  spatial  ability 
participants  probably  spent  more  time  and  effort  building  visual  representations  than  high  spatial  ability 
participants.  The  low  spatial  ability  participants  thus  had  fewer  cognitive  resources  available  for  building 
the  referential  connections  between  the  visual  and  verbal  information  in  the  CBI  program  (Mayer  & Sims, 
1994).  Consequently,  low  spatial  ability  participants  would  naturally  perceive  they  had  expended  a greater 
amount  of  mental  effort  than  high  spatial  ability  participants. 

Findings  for  time -in-program  and  time-in-instruction  indicated  that  participants  in  the  Animation 
treatments  took  significantly  more  time  (five  to  six  minutes  longer)  than  participants  in  the  Static 
Illustration  treatments.  This  difference  was  expected  and  supports  Baek  and  Layne’s  (1988)  finding  that 
animations  require  more  instructional  time  than  static  illustrations.  In  the  current  study,  the  eight  Animated 
instructional  sequences  took  9 minutes  and  20  seconds  to  play  assuming  no  sequence  was  replayed  by  the 
participant.  The  corresponding  eight  Static  Illustration  instructional  sequences  should  have  required  less 
time  for  the  participants  to  complete  as  they  need  not  have  waited  for  an  animation  to  finish  before 
proceeding.  In  contrast,  no  difference  in  terms  of  time-in-practice  was  found.  Participants  in  the  Animation 
treatments  spent  the  same  amount  of  time-in-practice  as  participants  in  the  Static  Illustration  treatments. 
Since  the  practice  screens  contained  only  static  illustrations  or  no  illustrations  at  all,  this  result  was 
expected. 

No  time  differences  of  any  type  were  found  for  participants  in  the  Text  versus  Audio  treatments. 
Again,  differences  for  time-in-practice  were  not  anticipated  since  the  practice  screens  were  the  same  in  the 
Text  as  in  the  Audio  versions.  However,  the  results  for  time -in-program  and  time-in-instruction  are 
puzzling. 

Participants  spent  the  same  amount  of  time  in  the  Text  as  in  the  Audio  treatments.  Based  on 
previous  studies  (Barron  & Kysilka,  1993;  Koroghlanian  & Sullivan,  2000)  and  common  sense,  one  would 
have  expected  the  Audio  version  to  take  longer  than  the  Text.  Examination  of  data  collected  during  CBI 
program  development  indicates  that  participants  in  the  Text  versions  should  have  completed  the  CBI 
program  1 0 to  15  minutes  sooner  than  those  in  the  Audio  versions.  This  time  difference  did  not  occur. 
Perhaps  participants  in  the  Text  versions  spent  those  10  to  15  minutes  rereading  the  text,  examining  the 
static  illustration/animation  or  trying  to  integrate  the  text  information  with  the  static  illustration/animation. 

Implications  for  Instructional  Design 

The  results  of  this  study  supports  previous  research  that  suggests  moving  some  text  from  the 
screen  to  audio  neither  hinders  nor  improves  learning.  This  finding  has  important  implications  for 
multimedia  development.  If  screen  “real  estate”  is  needed  for  something  other  than  instructional  text,  which 
is  especially  true  for  simulations  and  concepts  difficult  to  explain  with  words  alone,  then  text  can  be  moved 
from  the  screen  to  audio  with  no  loss  in  achievement.  This  is  an  important  and  useful  instructional 
technique  for  instructional  designers  to  consider,  especially  when  designing  materials  with  scientific  or 
technical  content. 

The  implications  of  this  study  are  less  straightforward  in  terms  of  animation.  Animation  did  not 
improve  learning  for  this  content  and  age  group.  Animation  did  take  more  instructional  time  than  static 
illustrations  with  no  corresponding  improvement  in  achievement  or  difference  in  attitude.  Whether  to 
include  animation  or  not  in  multimedia  programs  or  CBI  is  still  a matter  of  instinct,  not  research,  and  the 
final  decision  may  be  dictated  by  pragmatic  concerns  such  as  budget  or  time. 
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While  this  study  was  conducted  with  computer  based  instructional  materials;  the  results  have 
wider  implications  for  multimedia  instruction  in  general.  Web  based  instruction,  for  example,  increasingly 
incorporates  multimedia  attributes  such  as  audio  and  animation.  The  incorporation  of  these  attributes 
should  be  based  on  instructional  design  principles  and  research  to  ensure  effective  and  efficient  instruction. 

Suggestions  for  Future  Research 

Several  avenues  of  future  research  are  suggested  by  the  findings  of  this  study.  One  area  that 
warrants  further  investigation  is  the  physical  combination  of  audio  and  animation.  Some  researchers  might 
argue  that  the  present  study  did  not  minimize  the  split  attention  effect  and  thereby  did  not  optimize  the 
instruction  or  research  conditions.  Future  research  could  examine  superimposing  text  on  the  illustrations 
and  animations  as  well  as  utilizing  audio  only  with  illustrations  or  animations  followed  by  text  at  the  end  of 
the  sequence.  These  sequencing  and  layout  situations  would  tend  to  minimize  the  split  attention  effect  and 
might  clarify  research  results  and  subsequent  instructional  design  decision-making. 

One  puzzling  and  fascinating  result  of  the  present  study  concerns  the  activities  and  mental 
processes  of  the  participants.  Participants  spent  the  same  amount  of  time  in  the  Text  as  in  the  Audio 
versions  although  participants  in  the  Text  version  would  have  been  expected  to  finish  10-15  minutes  sooner 
than  those  in  the  Audio  version.  Interposing  questions  during  the  instruction  or  formally  observing 
participants  might  provide  information  of  use  and  interest  to  the  researcher  and  instructional  designer. 

Further  research  into  text  density  and  structure  would  be  valuable  to  instructional  designers 
designing  both  traditional  CBI  and  web-based  instruction.  Reducing  instructional  screen  text  while 
providing  the  majority  of  instruction  via  an  audio  track,  is  an  extremely  useful  technique  in  situations  of 
highly  complex  processes  and  simulations  where  there  is  a need  to  maximize  screen  space  for  non-text 
purposes.  Research  into  the  amount  of  text  required  when  text  is  combined  with  audio  and  the  manner  in 
which  that  text  should  be  structured  and  presented  warrants  further  investigation. 

Audio  and  animation  are  powerful  tools  for  the  instructional  designer.  Deciding  when  and  how  to 
use  these  tools  is  an  important  field  of  inquiry  that  deserves  more  attention  and  effort. 
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VIDEO-BASED  CLASSROOM 

Lessons  from  delivering  instruction 
In  an  interactive  video-based  classroom 
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Abstract 


Distance  education  is  growing  very  rapidly,  and  one  form  of  distance  education  is  two-way 
interactive  video . This  qualitative  pilot  study  presents  findings  from  interviews  conducted  with 
instructors  who  taught  in  a new  interactive  video  classroom . Instructor  responses  are  condensed 
and  summarized . Responses  include  instructor  insights  on:  student  perceptions;  instructor  skills 
required ; teaching  methods  used;  obstacles  to  overcome ; and  technical  considerations. 


Lessons  from  Delivering  Instruction 

In  an  Interactive  Video-Based  Classroom 

In  the  last  few  years  of  the  20th  century  institutions  of  higher  education  have  moved  toward  delivering 
instruction  via  distance  education  — the  separation  of  the  instructor  and  learner  by  time  and/or  location. 
While  distance  education  is  not  a new  phenomenon,  the  decreasing  cost  and  subsequent  ubiquity  of 
telecommunications  equipment,  computers,  and  networks  has  made  distance  education  a more  attractive 
option.  This  option  may  appear  favorable  to  innovative  instructors,  fiscally  conscious  administrators,  and 
students  who  cannot  attend  a residential  campus  (Moore  and  Kearsley,  1 996).  The  opportunity  to  reach 
new  educational  "markets"  has  encouraged  higher  education  institutions  to  pursue  various  modes  of 
distance  delivery.  One  such  delivery  mode  involves  two-way  interactive  compressed  video.  This  pilot 
study  will  describe  instructor  obstacles,  insights,  and  lessons  learned  from  course  delivery  in  an  interactive 
two-way  video  environment. 

Program  History  and  Description 

In  1998  the  University  of  West  Florida  began  course  delivery  via  two-way  interactive  video.  The 
University  constructed  two  classrooms  - one  at  the  main  campus  and  one  at  a branch  campus  75  miles 
away.  These  classrooms  were  equipped  with  VTEL  video  cameras,  microphones,  and  telecommunications 
equipment  to  enable  two-way  transmission  of  real-time  compressed  video  and  audio  signals.  This  system 
was  meant  to  replace  the  practice  of  instructors  commuting  to  and  from  the  main  and  branch  campuses  for 
teaching,  advising,  and  related  student  support  services.  The  student  population  at  branch  campus  was 
growing,  with  an  increasing  demand  for  courses  and  degree  programs.  Table  1 shows  a chronology  of 
significant  developmental  events. 


1993  UWF  Instructional  Technology  department  seeks  funds  for  video-based  classroom 

1997  Main  and  branch  classrooms  constructed  and  connected 

1998  Video-based  course  delivery  begins 

1 999  Web-based  course  delivery  begins 


Each  of  the  classrooms  was  entitled  the  Interactive  Distance  Learning  Studio  (IDLS),  as  a means 
of  identifying  the  unique  distance  education  environment  associated  with  the  classrooms.  Table  2 shows  the 
equipment  installed  in  the  two  IDLS  classrooms. 


Table  1 
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Table  2 


Two  cameras  — one  focused  on  the  instructor  and  one  focused  on  the  students 

Large  video  monitor  at  the  front  of  the  classroom  enabling  the  local  students  to  see  the  remote  students  (see 
Figure  1) 

Large  video  monitor  at  the  front  of  the  classroom  displaying  what  the  remote  students  were  seeing  (camera 
view)  (see  Figure  1) 

Large  video  monitor  at  the  rear  of  the  classroom  enabling  the  instructor  to  see  the  remote  students  (see 
Figure  2) 

Desk-based  microphones  for  each  student 

Microphone  and  camera  tracking  device  worn  by  instructor 

Twenty  networked  microcomputers 
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Figure  2:  View  from  the  rear  of  the  IDLS,  showing  the  remote  students 


The  initial  courses  offered  in  the  IDLS  were  taught  by  the  Instructional  Technology  (IT) 
department  faculty.  This  administrative  decision  was  based  on  the  premise  that  the  IT  faculty  would  be 
more  willing  and  motivated  to  use  the  telecommunications  technology  for  course  delivery.  Also  a 
facilitator  was  provided  for  each  interactive  video  course.  The  facilitator  supervised  the  remote  site  during 
each  class  meeting  and  provided  technical  and  instructional  support  to  the  course  instructor. 

Methods 

This  pilot  study  consisted  of  interviews  with  the  instructors  at  the  conclusion  of  the  first  semester 
of  classes  delivered  via  interactive  video.  The  interviews  were  all  conducted  by  the  author,  and  we  re 
designed  to  elicit  primarily  qualitative  information  about  the  teaching  experience.  The  interview  questions 
were  structured  as  follows: 

1 . What  technical  considerations  were  involved  in  teaching  in  this  environment? 

2.  What  is  required  of  students  in  this  environment? 

3.  How  did  the  facilitator  contribute  to  the  instructional  environment? 

4.  What  teaching  styles  or  strategies  did  you  employ  in  this  environment?  How  was  this  different  from  a 
conventional  course? 

5.  What  insights  or  recommendations  can  you  offer  to  instructors  teaching  via  interactive  video? 

Results  and  Discussion 

The  following  is  a condensed  summary  of  the  instructor  responses  to  the  interview  questions. 

1.  What  technical  considerations  were  involved  in  teaching  in  this  environment? 

In  terms  of  technical  considerations,  instructors  raised  several  issues.  The  need  for  practice  and 
skill  in  addressing  the  camera  (and  therefore  the  remote  site)  was  commonly  mentioned.  Addressing  a 
camera  while  teaching  was  an  unnatural  activity  and  required  behavior  modification  on  the  instructor's  part. 
Similarly,  interacting  with  remote  students  was  a challenge.  There  was  a perceived  lack  of  immediacy  in 
addressing  and  responding  to  the  remote  students  on  the  video  monitor,  as  opposed  to  the  physically 
present  local  students.  Also  instructors  needed  training  in  operating  and  switching  the  cameras  (from 
instructor  to  students)  via  the  touchpad.  Remembering  to  switch  the  views  at  appropriate  times  was  also  a 
challenge.  Instructors  had  to  manage  and  deal  with  possible  technical  difficulties  with  the  wearable 
tracking  microphone,  the  camera  touchpad,  the  document  camera,  and  a computer-based  presentation. 

2.  What  is  required  of  students  in  this  environment? 

Interaction  with  the  instructor  and  remote  students  were  the  key  issues  here.  In  this  type  of 
environment,  students  must  always  use  the  desk-mounted  personal  microphone  for  interaction  with  peers  or 

152 


the  instructor.  Otherwise  the  remote  site  is  excluded.  This  requirement  often  limited  or  silenced 
interaction,  particularly  short  comments  or  witticisms.  Short  comments  may  be  perceived  as  not  worth 
making,  weighed  against  having  to  wait  for  the  instructor  to  recognize  and  get  the  camera  switched  to  the 
student.  Students  had  to  adjust  to  being  on  camera  and  seeing  themselves  on  the  video  monitors. 

Whenever  a student  wanted  to  make  a comment,  he  or  she  had  to  be  prepared  for  the  camera  to 
automatically  turn  and  zoom  in  for  a close-up.  This  video  attention  may  have  discouraged  shy  students 
from  participating.  Interacting  with  distant  students  was  unnatural,  and  the  technical  and  time  requirements 
to  involve  remote  students  had  the  potential  to  create  resentment  with  the  local  students. 

3.  How  did  the  facilitator  contribute  to  the  instructional  environment? 

The  facilitator  evidently  is  a very  important  factor  in  determining  the  quality  of  the  experience  of 
the  remote  students.  It  is  highly  desirable  that  the  facilitator  has  subject  matter,  as  well  as  technical 
expertise.  The  facilitator  supports  the  remote  students  by  clarifying  parts  of  the  instruction,  distributing 
materials,  maintaining  order,  and  encouraging  interaction.  Interestingly  the  remote  students  may  develop  a 
closer  affiliation  with  the  facilitator  than  with  the  course  instructor,  and  this  must  be  considered.  The 
facilitator  was  often  a comfort  to  students  in  this  new  and  unfamiliar  environment.  For  remote  students,  the 
facilitator  is  perceived  as  "theirs". 

4.  What  teaching  styles  or  strategies  did  you  employ  in  this  environment?  How  was  this  different 
from  a conventional  course? 

Much  more  instructional  planning  was  needed,  including  planning  for  any  instructional  materials 
to  be  available  at  both  sites.  Entire  meetings  needed  be  scripted  like  a stage  play,  including  interaction 
within  and  between  sites.  Some  instructors  perceived  that  they  "needed  to  be  more  linear"  in  their  lecturing 
style,  due  to  the  video  medium.  A common  remark  was  that  it  was  much  less  effective  to  "wing  it"  — that 
is,  deliver  unplanned  or  less  formal  presentations.  Many  instructors  indicated  they  questioned  students  by 
name,  rather  than  using  open-ended  questioning  techniques.  Open-ended  questions  went  unanswered, 
possibly  due  to  reluctance  of  students  to  draw  camera  attention  to  themselves.  Instructors  attempted  inter- 
location  discussion  and  collaboration  via  video  in  class  and  asynchronous  Web  discussion  outside  of  class. 
Some  instructors  attempted  to  be  more  entertaining,  and  to  try  a variety  of  teaching  methods  to  maintain  the 
attention  of  the  remote  students.  One  instructor  perceived  that  he  was  less  mobile  (moving  throughout  the 
room)  due  to  the  limited  range  of  the  tracking  instructor  camera. 

5.  What  insights  or  recommendations  can  you  offer  to  instructors  teaching  via  interactive  video?  The 
following  represents  summary  statements  from  the  interviewed  instructors. 

• Students’  disdain  over  seeing  self  on  camera  led  to  decreased  participation  from  some  students,  and 
domination  of  discussion  by  others. 

• Using  student  microphones  destroys  spontaneity.  The  technology  suppresses  jokes,  clever  remarks,  or 
quick  interchanges.  (Is  your  witticism  worth  stopping  the  show?). 

• There  was  some  rivalry  and  potential  for  resentment  between  sites.  Wherever  the  live  instructor  taught 
was  perceived  as  the  “main”  or  “favorite”  group.  Larger  remote  class  size  (over  10)  makes  interaction 
difficult  (seeing  faces,  etc.) 

• In  unfamiliar  waters  it’s  good  to  appear  “in  command"  to  reduce  student  anxiety. 

• Travel  to  and  teach  from  the  remote  site  often  if  possible. 

• The  entire  class  meeting  must  be  scripted  like  a television  show. 

• Use  inter-site  collaboration/discussion  to  prevent  “us  vs.  them”  rivalries  from  developing. 

• Less  content  can  be  covered  than  in  a traditional  class.  Use  the  Web  to  continue  class  discussions. 

• Audio  and  the  interaction  dynamics  associated  with  it  are  the  most  important  technical  considerations. 
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• Lower-level  skills  may  predominate  (reading,  memorizing)  while  analysis  and  synthesis  are  difficult 
due  to  discussion  constraints  (microphones,  cameras). 

• Short  lectures  ( 1 0 minutes)  is  about  all  the  remote  students  will  tolerate.  (One  instructor  commented, 
“I’m  not  Tom  Cruise”,  meaning  he  was  not  a poiisned  video  entertainer). 

• The  instructor  must  spend  adequate  time  during  the  first  class  meeting  orienting  students  to  the 
environment  and  requiring  them  to  use  the  technology.  This  reinforces  the  required  interaction 
conventions. 

• Team  teaching  between  sites  requires  even  more  planning,  and  differences  between  instructors  can 
cause  difficulties. 
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Distance  Learning  and  Disability  Access:  A Success 


Kay  A.  Persichitte 
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University  of  Northern  Colorado 


This  paper  describes  a three  year  grant  project  (Federal  Grant  #H029A701 1 3)  at  the  University  of 
Northern  Colorado  to  design  and  deliver  a graduate  master’s  degree  program  in  blindness  and  visual 
impairment  to  the  14-state  region  of  the  Western  Interstate  Commission  on  Higher  Education  (WICHE). 
The  project,  funded  by  the  US  Department  of  Education,  began  in  January  of  1998.  The  Sl.l  M grant 
project  currently  offers  courses  to  78  students  working  to  complete  1 7 to  62  semester  hours  of  coursework 
(depending  on  previous  training  and  experience).  Additional  funds  have  been  contributed  by  private  and 
state  sources.  These  students  are  geographically  distributed  across  1 8 of  the  United  States,  over  half  of 
them  in  states  without  personnel  preparation  programs  in  blindness  and  visual  impairment.  The  first  classes 
were  delivered  in  the  fall  of  1 998.  Fourteen  courses  have  been  converted  for  distance  delivery  as  part  of 
this  project.  In  the  fall  of  2000,  eight  courses  were  delivered  with  a total  of  86  students  registered. 

While  students  who  are  blind  and  visually  impaired  (BVI)  represent  less  than  one-half  of  one 
percent  of  the  school  age  population,  they  are  a group  of  students  with  multiple  and  often  complex 
educational  needs.  One  of  these  needs  is  for  a specialized  teacher  trained  in  the  methodologies  ofblindness 
and  the  adaptations  necessary  to  facilitate  access  to  the  general  education  curriculum.  Rural  school  districts 
do  not  easily  meet  the  needs  of  these  students,  in  part  because  of  the  cost  of  hiring  these  specialized 
teachers,  and  in  part  because  qualified  teachers  of  students  with  visual  impairments  are  in  short  supply.  The 
shortage  grows  yearly  (Ingersoll,  1 999),  as  universities  close  teacher  preparation  programs  that  require 
significant  investments  in  fiscal  and  human  resources  without  producing  equivalent  tuition  revenues. 

Less  than  400  new  professionals  in  blindness  and  visual  impairment  enter  the  field  annually 
(Ferrell,  1999).  The  teacher  shortage  in  blindness  has  become  so  severe  that  the  Office  of  Special 
Education  Programs  funded  a special  project  to  investigate  the  depth  of  the  problem  and  to  develop  a 
national  plan  for  meeting  the  personnel  needs  of  the  future.  Although  the  results  of  this  project  have  not  yet 
been  published,  it  appears  that  the  nation’s  capacity  to  prepare  specialized  teachers  is  sorely  stretched. 
Using  technology  to  train  teachers  at  a distance,  especially  in  those  states  that  do  not  have  teacher  training 
programs  in  blindness,  is  one  way  of  expanding  the  nation’s  capacity  while  permitting  students  to  remain  in 
their  current  jobs  in  their  home  communities.  For  rural  school  districts,  this  may  be  the  only  way  they  will 
ever  recruit  a specialized  teacher  for  their  children  with  visual  impairments.  A particular  challenge  ofthis 
project  has  been  to  provide  a quality  graduate  education  experience  for  practicing  educators  who  cannot 
leave  the  special  needs  populations  that  they  currently  serve  to  re-tool  for  this  area  of  specialization. 

Project  Description 

At  the  culmination  of  the  three-year  funding,  this  project  has  increased  the  graduate  admissions  to 
the  Master’s  program  at  the  University  of  Northern  Colorado  (UNC)  over  five -fold.  Graduates  of  the 
program  each  year  now  outnumber  the  previous  cumulative  graduation  total  for  a four-year  period.  Seven 
faculty  have  participated  in  the  re-design  and  delivery  of  their  courses  using  eight  different  distance 
technologies  and  media.  Students  who  are  themselves  BVI  are  participating  and  we  have  recently  added  an 
instructor  who  is  blind  and  teaching  from  a remote  state. 

The  BVI  faculty  at  the  UNC  have  a deeply  held  philosophy  about  this  severe  needs  program.  It  was 
agreed  early  on  that  the  distance  delivered  program  would  subscribe  to  the  same  philosophy  and  that  has 
influenced  many  design  and  implementation  decisions.  The  philosophy  statement  is: 
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"The  UNC  Severe  Needs:  Vision  program  is  based  on  a firm  and  continuing  commitment  to  the 
rights  of  all  students  with  visual  and  other  disabilities  to  receive  equal  educational  opportunities,  including 
equal  access  to  the  curriculum.  The  faculty  believes  that  each  learner  should  be  provided  educational 
opportunities  that  maximize  potential  for  whatever  level  of  independence  is  possible  in  order  to  be 
productive  in  society  and  to  live  a meaningful  and  fulfilling  life." 

This  philosophy  has  become  a guiding  one  in  design  and  development  decisions  related  to  meeting 
the  needs  of  our  BVI  students.  The  concept  of  providing  “separate,  but  equal”  access  is  completely  contrary 
to  this  project.  While  it  may  be  more  difficult  for  the  person  without  sight  to  take  advantage  of  today's 
visual  distance  environments,  this  project  demonstrates  that  there  are  many  strategies  that  can  be 
incorporated  within  distance  learning  environments  to  leverage  the  communication  potential  of  these 
delivery  technologies  (see  “Building  Websites  for  the  Blind:  A Primer”  in  these  Proceedings).  A focus  on 
collaboration,  sharing,  and  contextualized  experiences  allows  not  just  "teaching-by-telling,  but  leaming-by 
doing"  (Stanard,  1999,  p.  49). 

This  project  is  one  example  of  Molly  Broad’s  comments  about  virtual  learning,  or  the 
“fundamental  importance  of  high-quality  faculty  and  effective  interaction,  both  between  faculty  and 
students  and  among  students.  Faculty  rightly  believe  these  are  fundamental  to  good  education;  however, 
with  the  growing  array  of  technology  tools,  it  is  possible  to  achieve  those  objectives  online.  In  addition, 
virtual  learning  can  also  bring  a very  rich  array  of  academic  resources  to  the  learning  process--resources 
that  address  the  multiple  learning  styles  of  students,  and  resources  that  greatly  enrich  the  educational 
materials  available  to  students"  (Morrison,  1998,  p.  3). 

The  project  team  consists  of  a Project  Director  and  Project  Coordinator  who  have  served  (along 
with  other  special  education  faculty)  as  subject  matter  experts,  a faculty  member  from  educational 
technology  who  has  served  as  the  primary  instructional  design  and  distance  delivery  consultant,  and 
multiple  graduate  students  from  educational  technology  who  have  served  as  designers,  developers, 
technical  assistants,  professional  development  coordinators,  and  as  remote  student  support  staff.  Adequate 
investment  in  both  human  and  technical  resources  is  one  reason  the  project  has  been  successful. 
Enrollments  are  now  stable  enough  to  sustain  the  delivery  of  the  distance  program  without  additional  grant 
funding. 

The  delivery  of  the  degree  program  is  grounded  in  a robust  web  environment  that  offers  content, 
additional  resources,  and  student  support  tools.  Every  course  includes  a website,  threaded  discussion,  class 
listserv,  and  synchronous  chat.  Other  customized  components  of  courses  include  web-based  interactive 
quizzes,  case  studies,  multimedia  tutorials,  customized  videotapes,  and  links  to  multiple  special  needs 
organizations.  In  addition,  there  is  a web- based  virtual  university  center  (listserv,  synchronous  chat,  bulletin 
board)  for  the  students  to  use  for  communication  and  collaborative  projects,  and  a threaded  discussion  area 
specifically  for  faculty  involved  in  distance  education  efforts.  The  development  of  a sense  of  community 
for  these  learners  and  faculty  has  been  the  recent  focus  of  project  refinement  and  elaboration  efforts 
through  scaffolding  communication  and  creating  a sense  of  place  for  remote  learners. 

Design  and  Development  Issues 

Instructional  design  (ID)  issues  that  have  influenced  the  project  cut  across  a broad  range. 

• alignment  of  course  content  with  four  sets  of  professional  standards 

• special  education  faculty  review  of  course  objectives  for  overlap  and  update 

• introduction/implementation  of  the  ID  process  (generically:  ADDIE  for  analysis,  design,  development, 
implementation,  evaluation)  with  special  education  faculty 

• helping  discipline  faculty  in  their  reconceptualization  and  adaptation  of  traditional  instructional 
strategies 

• delivery  system  and  media  selection  that  are  compatible  with  the  adaptive  technologies  used  by  BVI 
learners  (e.g.,  screen  readers,  braille  keyboards)  and  content  appropriate 

• materials  development  with  attention  to  the  special  needs  of  the  BVI,  but  not  to  the  exclusion  of  the 
creation  of  visually  stimulating  materials  and  environments  for  the  sighted  learners  and  instructors 

• discipline  faculty  preparation  and  support  as  they  teach  in  these  mediated  instructional  environments 

• complete  revision  of  student  assessment  and  evaluation  to  a standards-  and  performance-based  model 

• creation  of  student  and  faculty  support  materials 
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Other  issues  that  have  surfaced  are  related  to  the  administration  and  implementation  of  distance 
learning  programs. 

• faculty  and  student  access  to  distance  technologies  is  not  yet  ubiquitous;  high  quality,  dedicated 
technical  support  is  essential 

• importance  of  strong  administrative  support  from  the  College  of  Education  Dean 

• project  management  requirements  were  underestimated  (timelines,  coordination,  collaboration) 

• the  degree  program  is  complex  due  to  state  licensure  requirements  and  this  complexity  is  compounded 
when  students  participate  from  multiple  states 

• the  participation  of  non-special  education  faculty  requires  additional  time  and  design  support 

• other  campus  support  systems  (e.g.,  Academic  Technology  Services,  Registrar,  Scheduling)  must  be 
administrative  partners  in  such  large  scale  efforts 

• facility  design  was  required  (WWW  access  stations;  digital  video  development  station;  compressed 
video  classroom,  access  to  adaptive  hardware  and  software) 

• technical  considerations  at  the  development  level  and  the  end  user  level  (e.g.,  website  compatibility 
with  screen  readers,  software  versions,  Web  browsers  and  their  configuration;  software  downloads) 
must  be  addressed  simultaneously 

• remote  student  access  to  registration,  library  resources,  textbooks,  advising,  financial  aid,  and  other 
support  sendees  in  a university  environment  unprepared  for  these  requests. 

Delivery  Systems  and  Media 

The  project  purposefully  employs  a wide  variety  of  distance  delivery  systems  and  media.  In 
particular  instances,  materials  are  developed  in  more  than  one  media  to  allow  all  students  (sighted  and  non- 
sighted)  access.  Though  not  a stated  objective  of  the  project,  an  unintended  consequence  has  been  that  the 
students  are  increasing  their  use  of  and  comfort  with  technology,  in  general . All  members  of  the  project 
team  believe  in  the  power  of  technology  to  meet  leamerneeds  and  in  the  importance  of  better  preparing 
teachers  to  effectively  utilize  technology  with  their  students.  For  these  students  who  will  teach  children 
who  are  BVI,  Hardman’s  (1999)  comment  strikes  a loud  chord,  "A  technologically  competent  work  force  in 
the  education  industry  is  needed  to  continue  to  keep  the  promise  of  universal  education:  to  leave  behind  no 
child  who  is  willing  to  try"  (p.  4).  The  project  relies  on  the  WWW,  compressed  video  (CV),  text  (student 
handbooks  and  coursepacks),  videotape  (custom  and  commercial),  CD-ROM  (custom),  a required  campus 
component  during  one  summer,  computer  video  conferencing,  synchronous  and  asynchronous 
communication  via  the  Web,  audioconferencing,  and  commercial  satellite  downlinks. 

The  discipline  faculty  felt  strongly  that  the  distance  delivered  program  should  be  as  student- 
centered  as  the  campus  program.  The  design  and  development  process  has  consistently  incorporated  Sorg 
and  Truman's  (1997)  recommendations  for  creating  quality  student-centered  virtual  classes.  Their 
recommendations  included  personalizing  instruction,  humanizing  the  course  pages,  providing  advance 
organizers,  and  assuring  easy  navigation  between  and  among  course  topics.  During  the  grant  period,  the 
project  website  has  undergone  three  substantial  re-designs.  Each  one  has  brought  us  closer  to  the  desired 
student-centered,  interactive,  facilitated  distance  learning  environment  that  is  our  vision. 

Though  multiple  media  and  distance  systems  are  used  to  deliver  this  program,  the  WWW  is  the 
central  learner  and  instructional  resource  for  the  redesign  of  each  course  (http://vision.unco.edu/).  A 
standardized  navigation  shell  was  custom  created  so  students  do  not  feel  "lost"  each  time  they  begin  a new 
course  in  their  program.  Each  course,  however,  relies  to  varying  degree  on  the  Web  for  the  delivery  of 
instruction.  All  courses  have  embedded  syllabi,  links  to  the  four  sets  of  discipline  standards  and  course 
standards,  course  requirements,  description  of  course  activities,  an  asynchronous  threaded  discussion  area, 
a synchronous  discussion  area,  course  schedule,  and  a place  for  additional  resources  that  may  or  may  not  be 
web-based.  Each  course  also  has  a dedicated  class  listserv.  Some  of  the  course  websites  include  interactive 
custom-designed  tutorials,  samples  of  student  projects,  links  to  external  assistive  software,  and  multimedia 
authored  graphics.  The  project  website  is  not  password  protected,  but  all  course  sites  are.  The  variety  of 
technologies  in  use  has  increased  as  the  discipline  faculty  has  become  more  comfortable  with  trying  new 
instructional  strategies  with  remote  students. 

Remote  students  have  access  to  several  support  systems  that  have  proven  invaluable  to  the 
satisfaction  and  success  of  the  learners. 
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• online  Graduate  School  admissions  application 

• student  handbook  for  project  participants  (available  in  text  and  as  a PDF  file  on  the  Web) 

• toll-free  phone  into  the  Special  Education  Division  office 

• a Webmaster  who  responds  to  individual  technical  problems  with  near  24/7  response 

• CD-ROM  with  web  browser  and  style  sheet  options  to  load  on  home  computers 

• course  enrollment  and  program  enrollment  listservs  (subscribers  include  students,  faculty,  and  the 
grant  team) 

Lessons  Learned 

• Facility  design  of  distance  education  learning  environments  (DELEs)  is  expensive,  time  consuming, 
and  requires  substantial  technical,  pedagogical,  and  academic  expertise  related  to  distance  delivery  of 
instruction. 

• ID  and  FD  (facility  design)  need  to  evolve  simultaneously  for  DELEs  that  utilize  multiple  delivery 
systems/media. 

• Substantial  advance  planning  and  continual  project  management  is  critical  to  initiatives  of  this  scope. 

• Most  of  the  distance  delivery  technologies  today  are  visual  technologies;  consequently  there  is 
significant  attention  required  to  specialized  design  and  development  considerations  for  this  project  and 
for  any  other  distance  effort  that  strives  for  equal  access  for  disabled  learners. 

• Faculty  introduction  to  and  training  for  using  these  technologies  for  instructional  purposes  is 
particularly  important  to  project  success,  learner  satisfaction,  and  continued  faculty  involvement. 

• Meeting  individual  learner  needs,  faculty  expectations,  and  content  requirements  are  not  mutually 
exclusive  in  the  creation  of  a DELE,  but  the  process  is  extremely  complex. 

Future  Directions 

Federal  funding  ends  on  December  31 , 2000  and  the  staffhave  applied  for  additional  funding  to 
continue  the  project  and  keep  up  with  changing  technology.  Our  next  steps  include  technical  assistance 
regarding  online  course  delivery  to  other  universities  with  programs  in  visual  impairment  and  blindness,  as 
well  as  licensing  of  the  courses  for  delivery  at  other  universities.  Discussions  are  also  proceeding  related  to 
using  this  design  and  development  effort  as  a model  for  other  low  incidence  disability  degree  programs. 
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BVI  Resources 

Bobby  http://www.cast.org/bobbv/ 

EASI  at  Rochester  Institute  of  http://www.rit.edu/~easi 
Technology 
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“House  of  Style”  CSS  Tutorials 
IBM  Accessibility  Center 
Microsoft 

TRACE  at  University  of  Wisconsin- 
Mau  i son 

World  Wide  Web  Consortium’s  Web 
Accessibility  Initiative  (WAI) 

WWW  Consortium's  Synchronized 
Multimedia 

W3C  Validation  Section 


http://www.westciv.com/style_master/house/tutorials/index.html 

http://www-3.ibm.com/able/ 

http://www.microsoft.com/enable/ 

http://trace.wisc.edu/ 

http://www.w3c.org/WAI/ 

http://www.w3.org/AudioVideo/ 

http://www.w3.Org/MarkUp/#validation 
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Facilitating  Higher  Order  Thinking:  New  Teacher’s  Dilemma 


Vicki  S.  Williams 
Barry  O.  Williams 
Penn  State  Harrisburg 


Introduction 

International  interest  in  the  use  of  the  Internet  and  the  World  Wide  Web  (WWW)  in  learning 
environments  has  become  the  major  area  of  focus  for  many  educators  and  instructional  designers.  While 
few  dispute  the  value  of  hypermedia  in  instruction  and  training,  the  design,  development,  and 
implementation  of  web-based  learning  has  not  been  researched  in  any  exhaustive  way.  This  paper  will 
examine  the  experiences  of  one  design  and  development  team  attempting  to  encourage  critical  thinking 
skills  among  a group  of  pre-service  teacher  education  students. 

One  of  the  basic  components  of  the  course,  Higher  Order  Thinking  for  Educators,  a course 
required  of  all  students  majoring  in  Elementary  Education,  is  a series  of  case  studies.  The  cases  introduce 
the  students  to  problems  faced  by  teachers  in  public  schools  today.  The  course  currently  employs  five  case 
studies  to  allow  the  students  to  examine  a situation,  discuss  its  components,  and  arrive  at  possible  solutions 
to  the  scenario,  based  on  small  group  discussions.  The  learning  objective  forthe  cases  was  to  enable  pre- 
sendee  teaching  students  to  enhance  individual  problem  solving  abilities.  These  students  were  dispersed  in 
various  locations  within  the  state  in  clinical  experiences  and  spent  only  limited  time  in  class  with  the  cases. 
It  was  hoped  that  the  students  would  improve  their  higher-order  thinking  skills  and  problem-solving 
abilities  if  they  spent  more  time  working  with  the  cases  outside  of  class  and  then  be  able  to  transfer  what 
they  had  learned  to  new  situations  in  their  school  environments. 

Why  Case  Studies 

Case  studies  have  been  used  effectively  to  provide  scenarios  containing  problems  that  exist  in  real 
life  situations.  Since  successful  decision-making  is  based  on  experience,  and  many  students  lack 
experience,  case  studies  serve  to  simulate  experiences  and  enable  better  decision-making  for  novices. 
Kowalski,  Weaver,  and  Henson  (1990)  refer  to  case  studies  as  "..  .descriptions  of  a decision  or  a 
problem. . .normally  written  from  the  perspective  of  the  decision  maker  involved.  . . .Once  they  have  been 
developed,  cases  are  used  by  students  as  a way  of  putting  themselves  in  the  decision  maker's  or  problem 
solver's  shoes."  (p.  ix).  According  to  Wasserman  (1993),  using  cases  in  teaching  encourages  learning  by 
insisting  that  learners  and  teachers  deal  with  first-hand  knowledge  and  allow  students  to  think  critically 
about  the  situation,  making  sense  of  all  the  parts  of  the  case.  Then,  they  can  make  a truly  informed 
decision. 

Why  WBT? 

Considering  the  geographic  dispersion  of  the  students,  the  any  time,  any  place  nature  of  the  web 
offered  expanded  access  to  the  case  study.  It  also  provided  an  opportunity  for  students  to  self-pace  their 
learning  and  think  in  greaterdepth  about  the  situation  described  in  the  case.  Students  also  had  more  time  to 
reflect  on  the  information  they  encountered  and  uncovered.  To  encourage  this  increased  student-content 
interaction,  the  designers  used  digital  photos  of  the  individual  players  and  made  the  scenario  as  believable 
as  possible  turning  the  case  study  into  a simulation.  Using  images  and  text  together,  the  team  attempted  to 
create  a case  whose  “lessons  learned”  would  transfer  to  real  life.  It  was  important  that  the  students  could 
identify  with  and  put  themselves  in  the  "shoes"  of  the  new  teacher  portrayed  in  the  case. 

The  Design  Team 
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One  case  was  selected  for  conversion  to  web- based  delivery  and  was  designed  and  developed  by  a 
cross-discipline  team  consisting  of  two  students  and  four  faculty  members.  In  total  the  expertise  of 
instructional  designers,  a content  specialist,  and  technology  experts  were  integrated  to  design  and  develop 
this  unique  web-based  learning  experience.  The  two  instructional  designers  had  experience 
conceptualizing  and  designing  computer-based  instruction  and  WWW  applications.  Additionally,  each  had 
expertise  applying  pedagogical  theory  to  instructional  objectives.  The  course  instructor,  who  had  extensive 
experience  in  problem-based  learning,  provided  content.  On  the  technology  side  of  the  team,  both  student 
programmers  and  the  advising  Information  Systems  faculty  member  were  experienced  in  developing 
programs,  especially  utilizing  WWW-based  languages. 

Initially,  the  team  reviewed  the  case  to  make  certain  that  all  participants  understood  its  content  and 
the  objectives  for  the  activity  and  discussed  ideas  related  to  screen  design  and  navigation.  For  instance,  it 
was  agreed  that  the  online  version  must  be  creative,  engaging,  interactive,  and  utilize  motion  and  sound 
where  appropriate.  One  particularly  critical  activity  in  this  initial  stage  was  facilitating  the  content 
specialist's  realization  of  the  potential  of  the  technology.  While  she  had  experience  with  case-based 
problem  solving,  she  had  little  experience  with  the  WWW  and  associated  technologies.  No  attempt  was 
made  to  determine  exactly  how  concepts  would  be  presented,  since  in  this  particular  situation,  the  subject 
matter  expert  (SME)  was  also  the  client.  Thus,  it  was  imperative  that  we  educate  the  client  about  the 
capabilities  of  web-based  training  before  discussing  issues  such  as  video  clips  or  animated  gifs. 
Specifically,  it  was  critical  that  she  understand  the  nonlinear  nature  of  the  WWW  and  its  interactive 
capabilities. 

Design  and  Develop 

The  storyboard  (see  Figure  1 ) was  a funnel  for  our  brainstorming  sessions.  Capturing  the  essence 
of  these  sessions,  we  developed  a board.  In  addition,  we  adapted  a typical  storyboarding  situation  to  capture 
the  transformation  of  the  written  case  to  a proposed  screen  design  and  related  actions.  The  left  side  of  the 
storyboard  frame  is  a textual  excerpt  from  the  actual  case.  The  center  column  of  the  board  represents  the 
proposed  screen  design,  encompassing  both  graphics  and  text.  On  the  final  and  right  board  are  actions  to  be 
integrated  into  the  design. 

Figure  1 . Storyboard 
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Fill 

Storyboarding  and  graphic  representations  of  instructional  screens  and  site  maps  flowed  out  of 
brainstorming  sessions  and  into  the  prototype  product.  Using  various  tools  for  this  project  such  as 
Asymetrix©  ToolBook,  Hypertext  Markup  Language  (HTML),  JAVAScript™,  and  JAVA™  applets,  the 
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Rapid  Prototyping  phase  of  the  process  involved  an  actual  working  model  of  a portion(s)  of  the  storyboard. 
The  team  reviewed  the  prototypes  biweekly,  and  these  review  and  revise  sessions  resulted  in  the  client  / 
SME’s  having  a better  vision  of  where  the  website  was  headed  and  could  make  recommendations  for 
smaller  changes.  We  discussed  the  need  for  forthright  feedback  and  were  particularly  concerned  that  our 
student  developers  would  be  offended  by  faculty  critiques.  However,  this  concern  was  unfounded  and  the 
students  soon  felt  comfortable  debating  issues  during  these  sessions.  The  subgroup  of  student  programmers 
with  the  faculty  advisor  worked  to  produce  the  prototype  for  each  subsequent  review  session.  As  the 
website  developed,  animation,  sound,  and  interactive  segments  were  integrated  into  it,  bringing  the  product 
through  a startling  evolution  in  the  period  of  a year. 

The  Product 

The  final  product  was  a website  consisting  of  approximately  25  webpages  that  utilized  the 
strengths  of  the  WWW,  such  as  hypertext  links  to  resources  and  learner-directed  menu  choices.  They 
consisted  of  a home  page  (see  Figure  2),  acknowledgements,  instructions,  and  the  case  study.  The  case 
itself  was  divided  into  sections,  organizing  the  story  into  its  critical  chronological  stages.  In  the  first 
section,  students  are  introduced  to  the  new  teacher,  the  student,  the  principal  and  other  teachers  in  the 
school.  Demographics  were  available  on  the  school  itself,  but  the  learner  determined  the  order  of 
information  access  and,  in  fact,  whether  or  not  to  access  the  information  at  all.  While  making  the 
information  available,  it  was  up  to  the  learner  to  decide  what  information  they  needed  and  to  discriminate 
between  useful  and  nonuseful  information. 

Figure  2.  Getting  Acquainted 


Links  to  resources  such  as  student  records  and  school  guidelines  were  also  embedded  in  the  story 
and  the  learner  had  to  decide  to  access  the  information.  There  were  no  menu  links  to  these  resources,  as  a 
practicing  teacher  would  have  to  decide  with  in  a real  situation  what  information  they  might  need  to 
facilitate  their  decision-making.  The  client/SME  specifically  wanted  students  to  have  to  actively  investigate 
the  resources,  not  just  have  them  handed  to  them  in  the  menu. 

Student  remote  access  was  taken  into  consideration  and  any  motion  objects  were  animated  gifs 
rather  than  video  clips  to  eliminate  download  waits.  Sound  clip  file  sizes  were  kept  small  for  the  same 
reason  and  were  only  used  where  they  supported  the  content  or  were  used  as  an  object  of  interest. 

Results  of  the  Pilot 

A field  test  of  the  product  was  conducted  in  the  content  specialist’s  spring  semester  of  the  course. 
At  the  end  of  the  course,  Likert  Scale  attitudinal  surveys  were  completed  by  the  49  students.  The  feedback 
from  these  surveys  impacted  the  final  product,  which  was  implemented  the  following  academic  year.  The 
results  (see  Table  1)  showed  the  students  were  very  positive  about  the  instructional  value  and  aid  to 
retention  that  the  WWW  based  case  study  afforded  them.  They  liked  having  control  over  the  navigation 
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through  the  site  and  the  accessibility  of  the  instruction.  Through  the  WWW  they  could  spend  as  much  time 
as  needed  in  thinking  through  the  scenario  and  their  recommended  solutions  to  the  “Teacher’s  Dilemma” 
case.  In  addition,  the  client/SME  was  delighted  with  the  resulting  product  and  was  encouraged  to  think 
about  other  applications  of  technology  within  the  existing  curricula. 
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Table  1:  Student  Rankings  of  Key  Variables 


Student  Average  Ranking 

Content  of  Study: 

Memorable 

4.24 

Realistic 

4.47 

Thought  Provoking 

4.41 

Computer  as  a Medium 

Pictures  Added  Meaning 

4.59 

User  Friendly  Instructions 

4.12 

Ease  of  Navigation 

4.22 

Comparing  Paper  to  Computer 

Interesting 

4.22 

Enjoyable 

4.37 

Informative 

4.63 

The  scalability  of  this  project  indicates  the  process  can  be  successfully  replicated  in  the  remaining  case 
studies  of  the  course,  if  desired,  as  well  as  be  applied  to  similar  projects  in  other  disciplines  such  as  training 
and  development.  Providing  controlled  but  accurate  experiences,  while  still  allowing  for  safe  cognitive 
exploration,  would  encourage  trainees  to  examine  the  various  aspects  of  the  situational  simulation.  The 
cost-effective  nature  of  the  web-based  delivery  makes  this  an  attractive  alternative  to  paper  and  ink. 
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What  Software  Spiders  Can  Teach  Us  about  Collaboration 


Ruth  Gannon  Cook 
Joseph  Giarratano 
University  of  Houston  Clear  Lake 
Ken  Jenks 

Independent  Consultant 


Crossing  over  academic  disciplines  in  higher  education  is  seldom  done  unless  there  is  a 
compelling  reason,  such  as  the  information  technology  department  overlapping  into  the  social  science 
department  in  an  online  history  course.  Ironically,  different  departments  can  often  be  located  in  the  same 
building,  yet  the  walk  across  the  hall  from  one  college  to  another  can  be  a very  long  one.  Two  departments 
working  together  on  a project,  particularly  in  departments  like  the  College  of  Technology  or  the  College  of 
Education,  can  prove  more  difficult  than  the  biblical  expression  “passing  a camel  through  the  eye  of  a 


Occasionally,  however,  in  social  greetings,  conversations  can  ensue  that  have  a synergistic  effect 
and  grow  into  joint  projects.  Collaborative  efforts  develop  between  colleges  that  have  not  previously 
worked  together.  Such  is  the  case  in  this  study.  Two  instructors  from  different  colleges  in  a university 
casually  discussed  a research  project  that  could  benefit  both  departments.  They  would  create  a project  that 
would  combine  student  and  technology  resources  in  both  departments.  Once  the  idea  was  conceived,  the 
two  instructors  decided  upon  a topic,  a research  methodology  (qualitative),  then  embarked  upon  the  task  of 
assigning  the  project  to  students  in  the  fall  semester  of  1 999.  The  basic  concept  was  to  develop  a software 
"spider”  that  would  search  the  World  Wide  Web  to  detect  student  plagiarism  using  keyword  and  textual 
searches.  Such  spiders  and  other  software  tools,  like  intelligent  agents  and  knowbots,  are  commonly  used  to 
reduce  the  time  of  mundane  repetitive  Internet  tasks.  (A  good  example  of  such  a task  would  be  finding  the 
lowest  airfares  online,  or  looking  for  sites  with  the  lowest  mortgage  rates,  etc). 

Since  the  first  instructor  worked  in  the  Instructional  Technology  (IT)  Department  in  the  College  of 
Education,  she  was  familiar  with  both  curriculum  design  and  software  applications,  but  even  though  she 
taught  technology  applications  and  courseware  design,  none  of  her  students  had  actually  created  any 
software.  There  had  never  been  an  opportunity  to  design  or  test  a product  in  her  classes,  so  she  wondered 
how  that  process  could  be  experienced  with  students.  Moreover,  her  research  had  been  concentrated  on  the 
effects  of  semiotics  on  courseware  design  (GannonCook,  1999;  GannonCook  1998).  She  felt  it  would  be 
interesting  to  actually  design  a software  program  with  a metaphoric  theme  and  see  if  this  theme  facilitated 
recognition  for  the  functions  and  features  of  that  course.  She  had  been  searching  for  a way  to  create  both 
the  software  and  a way  to  test  the  importance  of  metaphor  in  software  design. 

The  second  instructor  worked  in  the  Computer  Science  (CS)  Department.  He  had  written 
numerous  textbooks  on  computer  instruction  and  supervised  graduate  students’  software  designs  for  many 
years.  One  of  the  more  interesting  courses  that  he  taught  actually  mentored  students  from  the  first  stages  of 
design  all  the  way  through  to  the  dissemination  of  the  product.  This  course  was  called  a capstone  course 
because  it  was  the  culmination  of  the  students’  learning  and  the  alternative  to  a Masters’  thesis.  The 
capstone  course  usually  included  outside  mentors  that  worked  with  students  so  the  students  would  get  a real 
world  understanding  of  development  teams.  Students  in  his  courses  have  worked  with  mentors  from 
NASA,  industry  and  business,  and  other  universities  to  develop  actual  products  instead  of  just  going 
through  classroom  programming  simulations.  His  software  projects  have  covered  a wide  array  of  products 
and  have  also  incorporated  website  design,  database  creation  as  well  as  various  types  of  hypermedia  into 
this  capstone  experience. 

During  one  of  their  friendly  “watercooler”  discussions,  the  second  instructor  mentioned  he  had 
students  who  could  work  in  internships  and  that  there  might  be  a student  who  would  fit  the  needs  of  the  IT 
instructor 's  dream  project.  After  several  additional  discussions,  they  decided  that,  indeed,  this  project 
might  be  one  that  could  be  meaningful,  not  only  for  this  study,  but  others.  If  research  could  be  conducted 
on  how  the  software  was  developed,  how  the  software  metaphors  were  selected,  and  how  the  software 
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application  functioned  as  an  Internet  spider,  it  could  provide  meaningful  data  to  both  colleges  within  the 
University  (Gallini,  Seaman  and  Terry,  1 995;  Verene,  1 993).  So  they  went  back  to  their  worlds  and 
worked  on  a research  plan,  then  reconvened  to  collaborate  and  combine  their  plans  (Bertalanffy,  1968; 
Bonk  and  Cunningham,  1998;  Bonk  and  King,  1998). 

A pleasant  surprise  to  both  instructors  was  that  their  plans  and  objectives  turned  out  to  be  very 
similar  (Ferris,  Roberts,  and  Skolnikoff,  1 997).  They  discovered  that,  despite  different  departments,  their 
philosophies  and  design  strategies  were  compatible  (Dillon,  1996;  Dillon  and  Gabbard,  1998).  They 
converged  their  research  objectives  into  one  plan,  which  they  set  out  to  implement  through  the 
collaborative  capstone  project  with  one  or  several  graduate  students  (Goss,  1 996;  Hirumi,  1 999;  Kauffman, 
1978). 

A timeline  was  created  and  a search  for  students  ensued.  Once  the  team  of  students  was  selected,  a 
third  person  who  had  volunteered  to  mentor  students  in  the  software  design  was  invited  to  join  the  team. 
Several  students  were  selected  to  participate  in  the  project.  Now  the  team  members  would  see  how  the 
ideas  could  be  developed  into  a product.  Hopefully,  they  would  also  observe  how  the  student  or  students 
created  the  software  program  and  how  the  student(s)  learned  to  work  with  others  in  a collaborative  team 
environment  with  other  group  members.  It  would  be  crucial  that  members  work  well  together  and 
communicate  status  of  each  phase  because  of  the  short  amount  of  time  allocated  to  create  and  complete  the 
project  (Havemik,  Messerschmitt  and  Vandrick,  1997). 

Several  students  opted  out  of  the  project  for  various  reasons,  so  it  distilled  to  one  student  who  was 
a very  proficient  programmer,  but  was  not  from  the  United  States  so  English  was  not  his  primary  language. 
He  had  no  experience  working  in  the  United  States  either,  so  it  would  be  a new  experience  to  work  with 
other  nationalities  in  any  kind  of  collaborative  environment.  This  student  was  assigned  to  work  with  the  IT 
teacher  and  was  told  to  embark  upon  his  course  with  “an  open  mind  and  a handy  notebook”. 

Once  introduced  to  the  IT  instructor,  they  had  a lengthy  discussion  about  the  instructional  design 
of  the  project  and  how  that  design  would  dictate  both  the  software  design  and  implementation.  The  original 
idea  had  been  to  design  a software  program  that  would  search  out  plagiarism  on  the  World  Wide  Web.  The 
collaboration  between  the  student  and  the  IT  teacher  would  be  an  experiment  too,  since  both  were  coming 
from  different  disciplines  and  backgrounds.  Fortunately  the  student  and  IT  teacher  worked  well  together, 
and  the  experiment  resulted  in  the  student’s  design  of  a spider  software  program  that  would  search  out 
plagiarism  on  the  World  Wide  Web  in  a number  of  search  modes. 

The  second  instructor  then  reconvened  with  the  first  to  discuss  and  review  how  the  collaboration 
between  the  IT  teacher  and  student  had  worked,  and  to  assess  the  success  of  the  software  design.  The 
overall  success  of  the  interdisciplinary  collaboration  would  be  gauged  by  the  functionality  of  the  software. 
But  the  ongoing  success  of  the  project  and  future  collaborative  projects  would  depend  on  the  ability  of  the 
team  members  to  work  well  together. 

The  student  worked  with  the  IT  instructor,  the  programming  mentor,  and  the  CS  instructor  and 
developed  a software  program  that  would  search  out  and  find  plagiarism  on  the  Internet.  He  created  Java 
servlets  for  the  web  server;  search  forms  and  software  to  act  as  spiders  that  would  catch  plagiarized  online 
works,  and  built  search  engines  that  would  search  for  classes  of  searchesby  author,  phrase,  and  direct  hits. 
Altavista  was  selected  as  the  commercial  search  engine  of  choice  to  utilize  the  spider  program  for  searches, 
and  the  spider  software  was  designed  to  send  e-mail  to  both  the  plagiarism  transgressor  and  the  creator  of 
the  plagiarism  search.  An  administrative  program  was  written  that  created  and  administered  a database  of 
information  extracted  from  the  web  searches.  Last  a program  was  written  (a  phrase  highlighter  program,  or 
PHP),  that  allowed  any  user  to  submit  to  submit  documents  using  HTML  forms  and  review  uploaded 
HTML  forms. 

In  this  instance  the  team  members  worked  well  together,  meeting  periodically  and  sending  each 
other  e-mails  to  report  the  progress  or  challenges  of  the  project.  The  results  were  the  creation  of  the 
program  that  provided  information  on  where  plagiarism  transgressions  existed  and  advised  searchers  of 
these  infringements.  The  project  went  well  and  the  team  members  discovered  that  some  of  the  most 
frustrating  problems  occurred,  ironically,  not  because  of  design  or  collaborative  efforts.  Most  of  the 
problems  revolved  around  server  hardware  and  software  problems.  Since  students  only  had  one  semester  to 
create  their  projects,  the  student  involved  in  the  spider  project  could  only  create  the  basic  program  and  had 
to  leave  improvements  to  be  made  in  future  capstone  projects. 
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Lessons  Learned  from  Working  with  Spiders 

The  creation  of  the  spider  project  provided  a product,  but  even  more  important,  it  created  a 
collaborative  web  for  all  of  the  participants.  It  interwove  threads  between  the  two  departments  that  had  not 
really  worked  together  prior  to  the  project;  it  involved  the  student  in  a real-world  scenario  with  deadlines 
and  diverse  corporate  partners;  and  it  created  an  actual  product  that  could  prove  useful  in  the  Internet 
marketplace.  The  IT  instructor  had  the  opportunity  to  participate  in  the  actual  creation  of  a product.  In 
addition  she  was  able  to  assist  in  creating  a product  that  used  a metaphor,  the  spider  web,  which  could  also 
be  assessed  for  relevance  and  user  recognition  (Lee,  1985).  Future  research  could  look  at  metaphoric 
content  and  research  the  effects  of  these  factors  on  the  ease  of  use  of  these  kinds  of  products. 

The  CS  instructor  had  the  opportunity  to  observe  how  his  capstone  student  worked  in  a 
collaborative  group  environment  with  both  corporate  mentors  and  another  department  within  the  university. 
He  felt  that  this  was  a particularly  beneficial  experience  for  the  student  because  the  university  mentor  was 
an  instructional  technologist.  She  had  provided  the  student  with  valuable  insights  on  how  to  prepare  his 
documentation  for  review  by  other  team  members  and  had  given  constructive  feedback  on  the  functional 
design  of  the  product.  The  CS  instructor  felt  that  future  collaborations  could  provide  even  more  insight  into 
team  project  development  and  hoped  additional  projects  would  emanate  from  this  effort. 

The  third  mentor,  the  entrepreneur,  was  excited  about  the  project  because,  as  a corporate  e- 
commerce  entrepreneur,  he  had  acquired  a potential  employee  to  develop  future  commercial  products  with 
the  help  of  interns  that  needed  the  real-world  experience  of  working  for  a corporation.  There  would  be 
opportunities  to  develop  grants  to  provide  to  the  university  by  corporate  sponsors  for  the  continued  creation 
of  projects  like  the  one  developed  in  this  study.  It  was  a good  incentive  for  the  mentor  to  consider 
additional  collaborative  efforts  with  the  university  on  future  projects. 

The  capstone  project  was  beneficial  to  the  student  because  it  not  only  gave  the  student  deadlines 
and  scenarios  similar  to  the  ones  he  would  encounter  when  he  graduated,  but  it  also  gave  him  the 
experience  of  working  collaboratively  in  a team.  He  created  the  product  required  for  the  satisfaction  of  the 
capstone  course  requirement  and  graduation,  but  walked  away  with  a better  understanding  of  what  it  was 
going  to  take  to  actually  communicate  his  ideas  and  programs.  The  student  provided  his  observations  to  his 
instructor  in  a journal.  Here  are  some  excerpts  of  his  final  thoughts  on  the  spider  project  experience: 

• Team  spirit  is  very  important  for  the  success  of  the  project. 

• Whether  the  project  succeeds  or  not  is  mostly  decided  by  the  client’s 
satisfaction  with  the  product. 

• The  skills  necessary  to  create  the  product  are  crucial,  but  so  are  the  presentation 
and  communication  skills  necessary  to  convey  what  needs  to  be  done  to  make  the  product 
possible  (Zhong  Xie,  1999). 

He  had  emerged  from  the  project  with  a respect  for  qualities  he  never  before  had  considered  to  be 
important  in  writing  programs  and  creating  software,  the  qualities  of  good  communication  and  reliability. 

The  outcome  surpassed  the  two  instructor’s  expectations.  The  project  created  by  the  collaboration 
had  cast  a web  that  reached  far  beyond  the  original  idea  formed  at  the  uater  fountain  between  the  two 
departments.  It  had  become  more  like  the  old  children’s  nursery  rhyme  of  the  “Itsy  Bitsy”  Spider.  The 
rhyme  describes  the  “Itsy  bitsy  spider  crawls  up  the  water  spout,  down  comes  the  rain  and  knocks  the 
spider  out.  Out  comes  the  sun  and  dries  the  spout  again,  so  the  itsy  bitsy  spider  goes  up  the  spout  again.” 
(Anonymous  nursery  rhyme) 

There  had,  indeed,  been  times  when  the  project  had  been  knocked  down,  then  the  team  and  student 
“climbed  up  the  spout  again”  by  trying  another  option  or  persisting  until  a solution  was  found.  The  result 
was  the  creation  of  a software  program  that  could  be  resilient  and  survive  downpours  of  everyday  virtual 
functionality. 

The  spider  proved  to  be  a good  metaphor,  demonstrating  resilience  and  casting  its’  web  to  catch 
intruders.  This  software  spider  taught  a memorable  lesson  about  respect  for  ownership,  but  more  important, 
it  taught  lessons  about  persistence.  Most  of  all,  the  biggest  lesson  may  have  been  learned  by  the  interaction 
and  participation  of  team  members  on  this  project.  Real  spiders  may  weave  solitary  webs  and  catch  prey, 
but  this  virtual  spider  wove  a web  that  was  strong  because  of  its  successful  team  collaboration. 
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Development  of  Program  on  Educational  Technology  in  China 


Meifeng  Liu 

Beijing  Normal  University 


In  comparison  with  that  in  other  countries,  Program  on  Educational  Technology > in  China  has  its 
own  characteristics  as  well  as  similarities  toother  countries.  This  paper  will  in  detail  introduce  evolution 
of  this  Program  in  China  as  well  as  its  current  status,  including  job  orientation  of  graduates  and  an 
organization  related  to  it.  Problems  existing  in  this  Program  will  be  qualitatively  analyzed.  At  the  end  of 
this  paper , some  ideas  about  the  Program  both  at  Beijing  Normal  University  in  China  and  Indiana 
University  in  U.S. A.  will  be  addressed. 

L Social  needs  related  to  the  profession  of  Educational  Technology  in  China 

A program  exists  because  there  must  be  some  social  need  for  it.  It  is  the  same  as  Program  on 
Educational  Technology. 

In  China,  there  is  a large  system  of  educational  technology  organizations.  It  includes  educational 
technology  in  K- 12  education,  named  Educational  Technology  Unit,  in  collegiate  education,  by  the  name  of 
Educational  Technology  Center,  as  well  as  in  society,  such  as  Educational  Technology  Center  or  Ministry 
in  bank  system,  in  medical  or  other  systems.  In  the  meantime,  many  Educational  Technology  Centers  exist 
in  central,  provincial,  city  as  well  as  county  levels.  Most  of  them  are  in  charge  of  development  of 
educational  technology  in  K-12  Education.  Also,  there  are  programs  in  universities  at  bachelor,  master  and 
doctoral  degree  levels.  Therefore,  professional  personnel  on  educational  technology  are  urgently  needed  in 
order  to  make  the  enterprise  of  Educational  Technology  run  productively.  It  is  widely  recognized  that 
Program  on  Educational  Technology,  which  the  professional  personnel  come  from,  is  vital  to  advancement 
of  the  whole  field  in  China. 

2.  Process  of  development  of  Program  on  Educational  Technology  in  China 

Simply  speaking,  there  are  two  stages  in  the  process  of  development  of  this  Program. 

(1)  Spontaneous  stage 

Educational  Technology  was  bom  in  1920s  in  China.  In  1930s,  it  was  developed  preliminarily. 
Program  on  Educational  Technology  in  China  originated  from  training  in  the  1930s.  At  that  time,  audio- 
visual media  were  applied  into  schools  and  universities.  There  was  need  for  professional  personnel  who 
could  operate  the  electronic  media.  Therefore,  related  seminars  were  organized  by  Ministry  of  Education 
as  well  as  courses  offered  by  some  universities,  for  example,  Da  Xia  University.  In  September  1936, 
Specialty  of  Film  and  Broadcast  Education  was  established  in  School  of  Education  in  Jiang  Su  Province.  In 
1946,  it  was  renamed  Specialty  of  Electrified  Education.  Also,  in  1938,  Specialty  of  Film  and  Broadcast 
Education  started  in  School  of  Science  at  Jin  Ling  University.  These  are  the  first  try  of  establishing 
Program  on  Educational  Technology  in  China. 

But  no  further  advancement  happened  in  the  later  forty  years  because  of  the  anti-Japanese  war, 
civil  war,  cultural  revolution  and  so  on.  At  the  end  of  the  1970s,  Program  on  Educational  Technology 
restarted  at  Hangzhou  University,  Zhejiang  Normal  University  and  Fujian  Normal  College  in  Department 
of  Physics.  At  that  time,  it  was  called  Program  on  Electrified  Education  because  it  focused  only  on  the 
application  and  maintenance  of  electronic  devices.  Before  the  1980s,  the  development  of  Program  on 
Educational  Technology  was  spontaneous,  that  means  there  was  no  central  governmental  layout  for  this 
Program. 

(2)  Unifying  layout  stage  by  Ministry  of  Education 

In  1979,  approved  by  Ministry  of  Education,  two  separate  Institutes  of  Modem  Educational 
Technology  were  built  in  Beijing  Normal  University  and  Hua  Dong  Normal  University.  They  did  a very 
good  job  to  start  the  exploration  of  Educational  Technology  in  China. 

In  November  of  1983,  a national  conference  was  held  on  educational  technology,  posing  a whole 
arrangement  for  the  establishment  of  Program  on  Educational  Technology.  From  then,  Program  on 
Educational  Technology  was  gradually  established  in  central  and  provincial  Teachers’  Universities  at  the 
bachelor  level.  This  is  the  first  time  for  Educational  Ministry  to  assign  all  the  issues  related  to  Program 
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building.  In  December  1986,  the  Committee  of  Degree  in  State  Department  approved  that  Beijing  Normal 
University,  Hua  Nan  Normal  University  and  He  Bei  University  could  award  Master  degrees  on  this 
Program.  Further,  in  1994,  Beijing  Normal  University  got  the  permission  of  offering  doctoral  degrees  from 
the  Committee  of  Degree  in  State  Department. 

Up  to  now,  there  are  more  than  30  universities  having  this  Program  offering  bachelor  degrees, 
about  15  universities  can  offer  master  degrees.  There  are  three  universities,  which  can  award  doctor 
degrees.  Also,  the  name  of  this  Program  has  gradually  changed  to  Program  on  Educational  Technology 
instead  of  on  Electrified  Education.  It  signified  the  focus  of  this  field  shift  from  hardware  to  software  as 
well  as  gradually  to  system  approach. 

An  academic  association  was  formed  in  China  in  1991  in  order  to  deal  with  Program  issues— 
Instructional  Material  Consult  Council  of  Program  on  Educational  Technology.  At  that  year,  it  held  a 
meeting.  During  that  meeting,  a unified  curriculum  and  syllabus  for  undergraduate,  which  was  developed 
by  Beijing  Normal  University,  passed  the  examination.  Since  then,  we  have  had  a unified  standard  for 
undergraduates  in  this  Program. 

This  Council  also  organized  and  compiled  many  textbooks  for  undergraduates.  Upon  completion 
of  these  tasks,  the  name  of  it  was  changed  from  Instructional  Material  Consult  Council  of  Program  on 
Educational  Technology  to  Instructional  Consult  Council  of  Program  on  Educational  Technology,  and  it  is 
responsible  for  research  issues  related  to  this  Program.  It  has  been  helping  to  pave  the  way  for  the  smooth 
development  of  this  Program. 

About  the  undergraduate  level,  in  1997,  we  conducted  a survey  about  the  Program  on  Educational 
Technology  in  Universities.  There  were  29  questionnaires  sent  out,  and  1 7 questionnaires  came  back.  Here 
are  some  results  about  this  survey: 

•Job  orientation  after  graduation  from  Bachelor  degrees 

From  the  highest  percentage  to  the  lowest,  the  graduates  with  bachelor  degrees  took  the  jobs  in  : 
A:  K-12  education  B:  collegiate  education 
C:  Educational  Technology  Centers  in  different  levels, 

D:  Academies  E:  others  F:  TV  Stations 

G:  Educational  TV  Stations  H:  Government  departments 

I:  Corporate  training  J:  Radio  and  TV  Universities  in  different  levels 
K:  Audio-Visual  publishing  press 

Among  those,  “others”  means  the  jobs  are  not  closely  related  to  the  Program  on  Educational 
Technology,  like  business  etc.. 

Figure  1.  Job  orientation  after  graduation  from  bachelor  degrees6 


6 The  data  came  from  a survey  analyzed  in  1997. 
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• Types  of  work  the  graduates  engaged  in 

The  graduates  would  engage  in  these  kinds  of  jobs  according  to  the  relative  percentages: 

A:  Instruction  B:  TV  program  producing  C:  TV  technique 

D:  Producing  Audio-Visual  instructional  materials  E:  Others 

F:  Audio- Visual  management  G:  Software  development 

H:  Research  on  Educational  Technology  I:  Gathering  and  editing  news. 

The  same  thing  as  the  former  figure,  “others”  means  their  work  is  not  related  to  the  Program  on 
Educational  Technology. 


7 The  same  as  footnote  1 . 

173 


685 


From  these  two  figures,  it  is  known  that  most  graduates  from  bachelor  degrees  engaged 
themselves  in  the  work  related  to  educational  settings.  Also  most  of  the  jobs  are  related  to  the  Program  on 
Educational  Technology.  But  there  is  still  a certain  amount  of  percentage  of  “others”  not  related  to  this 
Program. 

3.  Existing  problems  in  the  Program  and  Corresponding  strategies 

(1)  There  is  a problem  with  the  system  as  well  as  position  of  the  Program. 

Most  universities  put  this  Program  into  the  Department  of  Physics  at  the  very  beginning.  And  they 
gradually  became  independent  departments,  but  still  not  closely  associated  with  Educational  Departments. 
Also,  among  other  Programs  in  the  field  of  education,  few  have  recognized  this  Program’s  importance. 

(2)  There  is  a problem  with  professional  quality  of  faculty  in  this  Program 

Since  we  have  had  graduates  with  master  degrees  from  this  Program  in  1990,  with  doctoral 
degrees  in  1997,  most  faculty  members  in  this  Program  come  from  other  majors,  for  example,  physics 
major.  Also,  most  of  them  have  not  doctoral  degrees,  even  master  degrees.  The  quality  of  faculty  is  still  a 
problem  existing  in  this  Program  development. 

There  is  a high  need  for  graduates  with  master  or  doctor  rather  than  bachelor  degrees.  We  should 
have  more  master  and  doctor  students  registering  in  the  Program  on  Educational  Technology  rather  than 
bachelor  students. 

(3)  There  is  a problem  with  the  strategy  of  curriculum 

Technology  and  applying  technology  into  instruction  is  an  important  part  of  Educational 
Technology.  Also,  it  is  an  ever-changing  part  as  we  forge  into  the  information  society.  That  leads  to  even 
more  unstable  content  in  Program  on  Educational  Technology  than  that  on  Physics  and  some  other  majors. 
From  a curriculum  angle,  we  should  create  a balance  between  ever-changing  social  needs  and  stability  of 
the  curriculum  system.  We  need  to  design  a new  cumculum  strategy. 

Colloquium  or  other  workshops  should  formally  be  introduced  into  the  curriculum  system  in  order 
to  make  sure  that  students  can  have  the  latest  information  in  this  field. 

(4)  There  is  a problem  with  students’  real-life  experience  either  in  educational  settings  or  in 
corporate  settings 

Students  in  this  program  lack  real-world  experience  both  in  teaching  and  in  corporate  training 
settings.  Most  undergraduates  come  to  universities  directly  from  graduation  from  high  school.  So  it  is 
difficult  for  them  to  understand  some  of  the  courses  closely  related  to  teaching  experience. 

A bridge  should  be  built  between  the  Program  and  the  practical  educational  technology  field  in 
Educational  as  well  as  corporate  settings  in  order  to  offer  more  opportunities  for  students  to  intern. 

4.  Case  study:  Comparison  between  this  Program  at  Beijing  Normal  University  and  that  at  Indiana 
University 

Beijing  Normal  University  is  very  famous  for  education  all  over  the  country.  Program  on 
Educational  Technology  at  Beijing  Normal  University  is  also  a representative  for  this  Program  in  China. 
Here  are  some  ideas  about  the  comparison  of  this  Program  between  Beijing  Normal  University  and  Indiana 
University. 

(1)  About  the  system 

From  1985,  Program  on  Educational  Technology  had  been  one  of  three  Programs  in  the 
Department  of  Radio  and  Electronics.  Since  1999,  it  became  an  independent  department,  and  together  with 
three  other  departments,  Computer  Department,  Radio  and  Electronics  Department  and  Library  Science 
Department,  there  established  a college  named  Information  Science  College. 

Whereas,  Instructional  Systems  Technology  Department  belongs  to  School  of  Education. 

(2)  About  the  degrees  offered 

Beijing  Normal  University  can  offer  bachelor  degrees,  master  degrees  and  doctoral  degrees  on  this 
Program.  It  takes  students  four  years  to  get  bachelor  degrees,  three  years  for  master  degrees,  and  three 
years  for  doctoral  degrees.  But  if  a doctoral  student  is  only  a part-time  student,  He  or  she  can  spend  at  most 
five  years  to  finish  his  or  her  study.  Not  only  do  master  students  have  to  finish  certain  credits’  courses,  but 
also  they  should  have  to  write  a dissertation  with  the  same  process  as  doctoral  students’,  but  maybe  not  as 
involved  as  doctoral  students’. 
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1ST  at  Indiana  University  can  award  master  and  doctoral  degrees.  Master  degree  is  for  one  to  two 
years,  and  doctoral  degree  for  three  to  seven  years.  There  is  no  need  for  master  students  to  write  a 
dissertation,  but  they  have  to  compile  a portfolio. 

(3)  About  the  instruction 

Most  of  instruction  is  project-oriented  in  1ST  at  Indiana  University.  They  put  more  emphasis  on 
producing  knowledge  by  students  themselves.  Also,  through  project  approach,  team  building  and 
cooperative  ability  are  developed. 

As  to  instruction  of  this  Program  at  Beijing  Normal  University,  as  well  as  some  other  universities 
around  China,  we  usually  emphasize  imparting  knowledge  to  students.  We  encourage  dialogues  between 
students  as  well  as  between  teachers  and  students,  but  not  exactly  the  same  way  as  here. 

(4)  About  the  focus  of  Program 

Educational  Technology  Department  at  BNU  mostly  prepares  students  for  educational  settings. 
However,  1ST  Department  at  Indiana  University  focuses  on  both  educational  settings  and  corporate 
settings. 

In  China,  we  have  fewer  educational  software  companies.  Also,  the  training  issues  in  companies 
are  not  very  formally  conducted  except  for  foreign  or  joint-ventured  corporations.  So,  there  are  potentials 
for  China  to  develop  this  Program  in  Corporate  settings  in  the  future. 
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Current  Status  of  Learner  Support  in  Distance  Education: 
Emerging  Issues  and  Future  Research  Agenda 

Ji-Yeon  Lee 
Indiana  University 


Abstract 

The  importance  of  learner  support  has  been  emphasized  over  and  over  in  the  distance  education 
literature  but  there  is  a lack  of  comprehensive  analysis  of  this  issue . Besides  that,  most  of  existing 
studies  on  learner  support  are  based  on  large-scale  correspondence  program  and  few  of  them 
provide  guidelines  for  planning  and  implementing  support  services  in  web -based  environments  for 
dual-mode  institutions,  which  is  the  majority  of  distance  education  program  providers  in 
nowadays . In  this  paper,  a body  of  literature  on  learner  support,  including  various  journal 
articles  and  technical  reports,  was  examined,  and  three  issues  emerged  from  the  review:  (1)  the 
lack  of  research  on  cost-effectiveness  of  learner  support,  (2)  the  lack  of  empirical  research,  and 
(3)  the  need  for  learner-centered  approach  in  designing  and  implementing  learner  support . These 
issues  are  discussed  in  detail  to  enhance  our  understanding  of  learner  support  in  post -secondary 
level  distance  education  program,  and  it  is  further  argued  that  future  research  should focus  more 
on  developing  and  refining  methods  for  cost-benefit  analysis  of  learner  support,  developing  a 
general  framework  of  learner  support  model  in  dual -mode  institutions,  and  developing  systematic 
methods  to  identify,  analyze,  synthesize,  and  assess  the  needs  of  distance  learners . 


Introduction 

The  needs  for  learner  support  in  distance  education  comes  from  the  recognition  that  learning  in 
distance  is  often  mediated  by  networked  computers  with  individualistic  interfaces  that  require  learners  to 
work  alone  for  the  majority  of  learning  process.  Since  neither  the  instructor  nor  their  peers  are  physically 
present  to  help  and  direct  access  to  learning  resources  and  facilities  are  somewhat  limited,  distance  learners 
have  unique  needs,  and  many  researchers  and  practitioners  believe  that  providing  distance  learners 
appropriate  support  services  would  increase  the  quality  of  learning  (Feasley,  1983;  Gunawardena,  1988; 
Sahoo,  1993;  Watkins  & Wright,  1991).  It  is  not  surprising  that  learner  support  is  used  as  an  important 
accrediting  criterion  used  by  most  accrediting  agencies  (The  Institute  for  Higher  Education  Policy,  2000; 
Broad,  1999;  Mantyla  & Gividen,  2000). 

As  more  and  more  post-secondary  institutions  are  offering  distance  education  programs  or 
planning  to  do  so  in  the  near  future8,  there  is  a great  need  for  research  on  designing  and  implementing 
learner  support  services.  However,  the  majority  of  existing  studies  on  learner  support  are  based  on  large- 
scale  correspondence  programs  (i.e.  Open  University  in  U.K.),  and  few  of  them  provide  a comprehensive 
analysis  of  support  services  in  web-based  environments  or  guidelines  for  establishing  and  managing  learner 
support  systems  in  dual -mode  institutions  (Koble  & Bunker,  1997;  Simonson,  Schlosser,  & Hanson,  1 999). 

There  can  be  several  forces  underlying  the  gap  between  research  and  practice  of  learner  support, 
but  Robinson  (1995)  attribute  the  following  two  reasons  to  explain  the  situation:  (1)  learner  support  has 
been  perceived  as  peripheral  to  the  ‘real  business’  of  distance  education  which  is  developing  course 
materials,  and  (2)  many  researchers  tend  not  to  consider  learner  support  as  a suitable  topic  for  research 
since  it  is  contingent  on  local  circumstances  and  thus  not  easy  to  generalize  the  findings. 


8 According  to  a 1999  survey  by  the  National  Center  for  Educational  Statistics  (NCES),  nearly  1/3 
of  all  2-year  and  4-year  postsecondary  institutions  offered  distance  courses  in  1997-98,  and  additional 
twenty  percent  of  them  were  planning  to  offer  distance  courses  within  three  years. 
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His  points  may  have  been  relevant  in  the  past  when  distance  education  was  viewed  as  inferior 
alternative  to  traditional  face-to-face  education  and  the  emphasis  was  on  the  product,  rather  than  the 
process.  But  things  are  changing,  if  not  already  changed,  and  distance  education  is  considered  as  a major 
phenomenon  in  higher  education  these  days.  Distance  learners  are  much  more  sophist  seated,  ui  versified, 
and  demanding  than  ever,  and  they  expect  a lot  more  than  well-designed  learning  material.  Besides  that, 
the  competition  among  distance  education  providersis  such  that  if  an  institution  fails  to  satisfy  the  students, 
it  will  loose  them  to  one  of  its  competitors.  Thus  many  institutions  offering  distance  education  programs 
are  struggling  to  better  meet  the  needs  of  their  students.  Unfortunately,  research  on  learner  support  at  this 
point  is  not  able  to  provide  much  guidance  for  the  institutions  to  cope  with  these  problems,  and 
practitioners  are  learning  the  lessons  in  a hard  way  - by  trials  and  errors. 

After  reviewing  the  literature,  three  issues  emerged  as  most  problematic  areas  in  research  on 
learner  support:  (1)  the  cost-effectiveness  of  learner  support,  (2)  the  lack  of  empirical  research  and  the 
difficulties  in  generalizing  research  findings  as  a result,  and  (3)  the  need  for  learner-centered  approach.  I 
believe  tackling  these  issues  with  greater  depth  will  help  enhancing  our  understanding  of  learner  support  in 
distance  education  and  also  help  advancing  research  in  this  area.  Thus  the  purpose  of  this  paper  is  to 
review  current  status  of  research  on  these  three  issues  and  suggest  recommendations  for  future  research  on 
learner  support  in  distance  education. 

Definition  of  Learner  Support 

Defining  the  key  elements  and  boundaries  of  learner  support  is  important  in  research  since  it 
provides  a criterion  in  determining  the  standard  or  quality  of  learner  support.  However,  learner  support  is 
rather  a broad  concept  and  its  definition  varies(Robinson,  1 995;  Sewart,  1 993).  Some  researchers  consider 
resources  and  interactivity  as  critical  in  defining  learner  support  (Garrison,  1 987)  while  others  put  more 
emphasis  on  individualization  or  customization  of  services  (Thorpe,  1988;  Tait,  1 995). 

There  are  two  distinctiw  views  to  approach  learner  support  - supplementary  and 
complementary/holistic  (Robinson,  1995;  Tait,  1995;  Rowntree,  1992;Nunan,  1993).  The  former  is  more 
limited  in  that  learner  support  is  confined  as  an  add-on  to  course  materials  or  other  learning  experiences 
while  the  latter  view  it  as  a crucial  factor  which  pervades  the  entire  education  program  and  something 
without  which  distance  students’  learning  experience  cannot  be  complete. 

It  seems  that  more  and  more  researchers  are  taking  a holistic  approach  considering  learner  support 
as  an  integrated  part  of  course  and  the  entire  learning  process(Sache  & Mark,  2000;  Scalzo,  Matela-Rodier, 
& Ferrauilo,  2000).  From  complementary  perspective,  learner  support  is  all  about  providing  access  to  both 
resources  and  opportunities  that  leads  to  lifelong  learning  (Reid,  1995;  Smith,  2000).  It  extends  the  range 
and  duration  of  services  and  emphasizes  the  importance  of  providing  quality  information,  advice  and 
guidance  at  pre-enrollment  and  early  post-enrollment  stages.  Hardy  & Boaz  ( 1 997)  even  extend  the 
concept  of  learner  support  into  the  next  level  - “learner  development”  meaning  preparation  of  the  learners 
for  a distance  learning  experience  beyond  the  technical  assistance. 

What  are  the  Elements  of  Learner  Support? 

There  is  an  almost  infinite  variation  in  learner  support  systems  in  distance  education,  and  as 
Sewart  (1993)  commented,  each  system  is  unique  in  a sense  that  it  is  dealing  with  a different  student 
population  in  a different  context.  Thus  developing  a general  yet  representative  framework  of  learner 
support  has  been  a challenging  task  for  researchers  in  this  area. 

One  of  the  most  comprehensive  lists  of  elements  of  such  a system  was  developed  by  Keast(Keast, 

1 997).  He  identified  four  distinctive  types  of  support  for  distance  learners  - administrative  support, 
instructional  support,  technical  support,  and  counseling/tutorial  support.  This  list  is  by  no  means 
exhaustive  and  misses  library  support,  a very  important  category  that  is  gaining  growing  attentions  these 
days.  However,  it  still  captures  key  functions  of  learner  support  and  most  support  services  suggested  by 
other  researchers  or  practitioners  fall  under  Keast’s  categories  (Aoki  & Pogroszewski,  1998;  Frieden,  1999; 
Reid,  1 995;  Sache  & Mark,  2000;  Tait,  1 995). 
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Each  category  will  be  further  elaborated  in  the  following  section,  and  considering  the  special 
relevance  to  current  emphasis  on  informational  technology  in  distance  education,  library  support  will  be 
added  to  Keasf  s four  elements  of  learner  support  system. 

Academic/Tutoria!  Support 

Academic/tutorial  support  is  largely  based  on  Open  University  model  in  UK.  where  students  have 
access  to  local  study  centers  and  tutors  who  supervise  their  academic  progress  and  help  with  problems 
(Watkins  & Wright,  1991;  Sahoo,  1993;  Rae,  1989).  In  more  recent  web-based  distance  education 
programs,  the  focus  of  academic  support  gears  toward  facilitating  collaborative  learning  and  increasing 
interactivity  between  distance  students  and  instructor  or  among  distance  students  (Aoki  et  ah,  1998).  Some 
of  the  examples  of  such  services  include  syndicate  or  learning  groups,  support  by  instructor  on  request, 
workshops  to  assist  students  to  develop  specific  skills  or  bridge  skill  gaps,  supervision  support  on  research 
project,  and  “learning  contract”  and  scheduling  (SAIDE,  1998). 

Administrative  Support 

Administrative  support  services  involve  maintaining  basic  program  functions  such  as  admissions, 
registration,  course  scheduling,  student  records,  and  financial  transactions  (Frieden,  1999).  These  services 
are  often  taken  granted  but  when  not  planned  carefully,  they  cause  greatest  frustration  from  distance 
students. 

Technical  Support 

Technical  support  refers  to  monitoring  efficient  operation  of  delivery  mediums  and  offering 
technical  assistance.  While  much  research  has  been  conducted  on  the  use  of  new  technologies  in  designing 
and  developing  distance  courses,  less  effort  has  been  directed  toward  the  use  of  new  technologies  to 
provide  support  services  for  such  courses  (Abate,  1999).  Providing  an  800  telephone  number  for  students 
to  contact  the  faculty  and  staff,  requiring  that  all  students  have  access  to  email  and  know  how  to  use  it, 
requiring  faculty  to  schedule  office  hours  particularly  for  distance  students  at  times  that  would  be 
convenient  to  the  students  (Reinert  & Fryback,  1997),  and  it  may  be  necessary  to  devote  additional  on- 
campus  facilities  to  support  the  off-campus  population. 

Counseling  Support 

Counseling  support  includes  various  aspects  of  guidance  and  advising.  In  correspondence  studies 
or  other  delivery  medium  with  more  individualistic  interface,  the  focus  of  such  services  tends  to  be  focused 
on  how  to  deal  with  academic  concerns  and/or  career  advising.  In  many  web-based  distance  education 
programs,  counseling  support  also  deals  with  ways  to  improve  communication  skills  and  increase 
interactivity,  and  even  network  with  alumni  and  community  building  (Aoki  et  al.,  1998). 

Many  institutions  are  also  requiring  orientation  sessions  that  bring  the  students  on  campus  in  order 
to  familiarize  them  with  the  services  that  are  available.  Such  sessions  would  provide  an  opportunity  to  learn 
the  interfaces  used  to  access  the  services  as  well  as  a chance  to  interact  with  the  support  personnel  on- 
campus  (Thompson,  Winterfield,  & Flanders,  1998). 

Library  Support 

Access  to  adequate  library  services  and  resources  is  essential  for  the  attainment  of  superior 
academic  skills  in  post-secondary  education  and  distance  learners  are  entitled  to  library  services  and 
resources  equivalent  to  those  provided  for  students  in  traditional  campus  settings.  However,  traditional  on- 
campus  library  services  often  fail  to  stretch  themselves  to  meet  the  library  needs  of  distance  students. 

The  Association  of  College  and  Research  Libraries  (Libraries,  1998)  provides  a guideline  for 
distance  education  programs  to  ensure  that  library  support  meet  the  students’  needs  in  fulfilling  course 
assignments  (e.g.  required  and  supplemental  readings)  and  accommodate  other  information  needs  as 
appropriate.  Some  of  the  specific  examples  of  library  services  to  meet  those  needs  include  region- wide 

179 


691 


borrowers  cards,  consortia  membership  between  academic  libraries,  and  fax  and  online  capabilities  for  the 
timely  document  delivery  (Aoki  & Pogroszewski,  1998). 

In  a review  of  the  literature  on  distance  learning  library  support,  Stephens  (Stephens,  1996) 
stressed  that  what  was  lacking  was  not  only  books  and  journals  per  se,  but  instruction  and  the  opportunity 
to  do  independent  'library  research.  To  be  able  to  fill  this  gap,  distance  library  services  need  to  be  more 
customized  and  empowering  distance  learners,  such  as  providing  toll-free  telephone  numbers  for  library 
help  desk  and  capabilities  to  use  multiple  databases  and  online  public  access  catalog  (Aoki  & 
Pogroszewski,  1 998).  The  combination  of  special  funding  arrangements,  proactive  planning,  and  promotion 
is  necessary  to  provide  such  services  (Smith,  2000). 

Emerging  Issues  in  Learner  Support 

Many  researchers  expected  that  advances  in  technology  would  make  it  easier  to  provide  quality 
support  services  with  increased  interactivity  and  automatization  (Bates,  1994).  However,  for  all  those 
dramatic  changes  in  terms  of  delivery  medium,  from  correspondence  to  audio  and  video  conferencing 
systems,  then  to  the  Internet  in  nowadays,  many  issues  are  remained  as  same,  if  not  technology  adds 
another  level  of  complexity.  Some  of  them  are  more  critical  and  have  greater  implications  for  future 
research  than  others,  and  thus  they  will  be  discussed  in  more  detail  in  the  rest  of  this  paper.  Those  issues 
are:  ( 1 ) the  cost-effectiveness  of  learner  support  services,  (2)  the  lack  of  empirical  research  and  difficulties 
in  generalizing  the  research  findings  as  a result,  and  (3)  the  importance  of  learner-centered  approach  in 
learner  support. 

Cost-Effectiveness  of  Learner  Support 

The  first  issue  is  related  to  how  to  scale  customized  support  services  in  a cost-effective  way.  In 
order  to  better  meet  the  diverse  needs  of  distance  students,  support  services  needs  t o be  more  individualized 
(Brent,  1 995;  Sahoo,  1 993).  Providing  such  services  is  not  an  inexpensive  proposition,  however,  and  the 
underlying  assumption  is  that  the  greater  the  input  to  the  provision  of  learner  support  services,  the  greater 
the  completion  rate  and/or  learning  outcomes  (European  Commission,  1 996). 

At  this  point,  there  are  few  studies  that  offer  guidelines  on  this  area,  and  most  of  them  are  focused 
on  cost  analysis  of  technology  infrastructure  (Brent,  1 999;  Rumble,  1 993,  1 999;  Whalen  & Wright,  1 999). 
Although  those  cost  estimation  studies  do  offer  policymakers  some  insight  into  the  types  and  range  of  costs 
associated  with  distance  education  program  in  general,  we  can  only  infer  from  this  broader  framework  how 
much  it  will  cost  to  provide  certain  support  services. 

One  of  the  biggest  challenges  in  approaching  the  cost-effectiveness  of  leaner  support  is  that  the 
relationship  between  the  input  and  output  is  not  straight-line  equation,  and  as  the  level  of  support  gets 
beyond  a certain  point,  the  curve  of  student  success  seems  to  be  flatten  out  (Sewart,  1993).  The  fact  that 
there  are  relatively  few  references  to  direct  and  indirect  costs  involved  with  various  support  services  also 
make  it  even  more  difficult  to  measure  the  cost-effectiveness  of  certain  support  services  (Tait,  1995; 
Wagner,  1999). 

The  issue  of  cost-effectiveness  in  providing  learner  support  services  is  also  related  to  the  mission 
of  institution.  The  range  and  standard  of  learner  support  service  provided  by  an  institution  will  be 
eventually  determined  by  whether  it  is  more  concerned  about  services  or  making  profits  from  offering 
distance  education  programs.  The  bottom  line  is,  in  any  case,  the  basic  services  needs  to  be  provided 
regardless  of  the  costs  to  guarantee  the  quality  of  education,  and  for  beyond  that,  it  is  totally  up  to  each 
institution  to  decide  whether  or  not  to  offer  more  services. 

Cost-effectiveness  of  learner  support  is  likely  to  be  achieved  when  the  support  system  is  structured 
to  do  “more  with  less  ” and  this  requires  some  creativity  for  institutions.  One  such  example  of  scaling 
student  services  is  partnering  with  other  organizations  such  as  businesses  and  school  districts  (Hickman, 

1 999).  This  may  be  an  appropriate  method  for  providing  quality  services  by  providing  broader  access  and 
cutting  the  expenses  on  support  staff  training  and  maintenance  by  delegating  the  services  to  the  specialists. 

Lack  of  Empirical  Research  and  Difficulties  in  Generalizing  the  Findings 
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The  second  issue  revolves  around  the  dominance  of  pragmatism  in  research  of  learner  support. 
There  are  almost  infinite  variations  of  learner  support  in  distance  education  practice,  and  the  choice  and  use 
of  certain  services  among  those  numerous  services  is  largely  based  on  practicalities  rather  than  research 
findings.  This  leads  to  a tendency  that  pragmatism  outweighs  empirical  inquiry  or  research  in  the  area  of 
learner  support,  which  is  confirmed  by  Robinson  ( 1 995).  Based  on  the  review  of  previous  research  and 
practice  in  this  area,  he  concludes  that  learner  support  is  heavily  contingent  on  local  circumstances.  There 
is  nothing  wrong  with  research  informed  by  practice,  and  the  contingent  nature  of  learner  support  seems  to 
prone  to  such  ‘how  to  do  it’  type  of  case  studies.  However,  there  are  missed  opportunities  where  we  can 
further  exploit  this  critical  issue  in  distance  education. 

Christenson  (1998)  made  an  interesting  argument  on  how  empirical  research  in  social  sciences 
may  be  used  to  guide  practice  as  a framework.  In  developing  theory  in  the  social  science,  researchers’ 
desires  to  extend  the  application  of  the  best  observed  practice  often  leads  them  to  emphasis  too  much  on 
observation  with  the  expense  of  theoretical  reasoning.  I believe  this  may  be  one  of  the  major  problems 
associated  with  pragmatic  research  in  distance  education,  since  in  a case  study,  the  generalizability  of 
findings  and  its  predictive  power  is  somewhat  limited  to  the  institutional  boundary  where  the  practice  is 
emulated.  Thus  this  type  of  research  often  advocates  the  status  quo  rather  than  advancing  theories  and 
seeking/predicting  changes  beyond  what  has  been  observed. 

Each  support  system  is  unique  in  a sense  that  it  is  dealing  with  a different  student  population  in  a 
different  context,  of  course,  but  even  so,  we  can  always  use  of  a general  framework  or  model  for  learner 
support  that  is  grounded  in  learning  theories  and  supported  by  empirical  data.  Without  such  a framework, 
every  institution  needs  to  learn  what  does  and  does  not  work  for  them  at  expensive  cost  - trials  and  errors. 
And  it  would  not  just  the  institution  but  also  the  distance  students  that  have  to  pay  the  cost. 

Need  for  Learner-Centered  Approach 

Last  issue  is  related  to  the  need  for  learner-centered  or  customer-focused  approach  in  planning  and 
implementing  learner  support.  No  one  can  understand  the  difficulties  that  distance  learners  encounter 
better  than  the  learners  themselves.  Yet  in  many  institutions  offering  distance  education  programs,  learner 
support  is  based  on  top-down  provision  rather  than  analysis  of  learners’  needs  (Tait,  1995;  Sache  & Mark, 
2000;  Scalzo  et  al.,  2000).  It  is  not  new  to  the  field  of  learner  support  at  all,  and  had  already  been  clearly 
stated  by  D.  Sewart  (1987): 

It  does  not  seem  unfair  to  suggest  that  there  is  an  overwhelming  tendency  within  the  field  to  offer 
systems  from  the  viewpoint  of  the  institution  teaching  at  a distance  rather  than  from  the  viewpoint  of  the 
student  learning  at  a distance  (p.72). 

Traditionally,  education  has  represented  a provider-led  rather  than  a customer-led  activity  and  the 
central  question  of  identifying  student  needs  were  often  neglected.  In  the  past  when  distance  education  was 
viewed  as  a product,  rather  than  a process,  and  the  quality  of  learning  was  identified  as  the  quality  of 
learning  material,  institutions  were  able  to  operate  successfully  with  the  provider-led  mindset. 

However,  things  have  been  changed,  and  today’s  distance  learners  are  much  more  sophisticated, 
diversified,  and  demanding  than  ever,  and  they  expect  a lot  more  than  well-designed  learning  materials. 
Besides  that,  the  competition  among  distance  education  providers  is  such  that  if  an  institution  fails  to 
satisfy  the  students,  it  will  loose  them  to  one  of  its  competitors.  Understanding  learners  are  critical  in 
providing  appropriate  support  services  for  the  survival  of  distance  education  institutions  nowadays. 

When  incorporating  a learner-centered  approach  in  designing  and  implementing  learner  support, 
we  have  to  understand  that  it  is  a continuous  process,  rather  than  one-shot  activity.  Different  learners  may 
have  different  needs,  and  those  needs  may  be  changed  over  time.  Those  differences  or  changes  can  only  be 
traced  by  systematic  and  continuous  efforts  to  identify,  analyze,  synthesize,  and  assess  distance  learners’ 
needs. 

Nunan  (1993)  suggests  a user-pay  system  as  a ways  to  meet  diverse  user  needs  and  expectation. 
He  believes  that  by  generating  choices  and  options  that  can  be  purchased  according  to  the  individual  needs 
and  interprets,  an  institution  can  achieve  customization  in  a more  cost-effective  way.  This  can  be  an 
insightful  resolution  approaching  learner-centered  support  services,  but  if  it  is  not  followed  by  accurate 
descriptions  of  each  option  and  counseling  services,  the  learner  may  not  be  able  to  make  informed 
decisions  to  select  what  is  best  for  themselves.  Also,  the  institution  should  provide  some  of  the  most  basic 
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services  such  as  library  support  or  technical  support  as  default.  Otherwise,  it  may  sacrifice  the  quality  of 
education  at  the  expense  of  convenience  or  cost-effectiveness. 

Suggestions  for  Future  Research 

Learner  support  in  distance  education  needs  to  be  justified  not  only  pedagogically  but  also 
financially,  and  to  be  able  to  do  so,  research  on  learner  support  should  support  the  following  areas:  (1) 
developing/refining  the  methods  of  cost-benefit  analysis,  (2)  developing  a general  model  of  learner  support 
in  web-based,  dual-mode  institutions,  and  (3)  providing  a ways  to  systematically  incorporate  the  needs  of 
distance  learners  in  designing  and  implementing  learner  support  services. 

There  have  been  a lot  of  claims  that  providing  appropriate  learner  support  services  would  increase 
the  quality  of  learning  and  would  positively  influence  student  retention  and  satisfaction.  However,  these 
claims  were  rarely  accompanied  by  supporting  data,  which  leads  to  the  tendency  that  learner  support 
services  become  vulnerable  to  financial  fluctuation.  The  only  way  to  deal  with  this  issue  is  to  come  up 
with  sound  methodologies  to  measure  the  cost-benefit  of  learner  support. 

The  framework  suggested  by  Cuckier  (1997)  is  promising  in  that  it  includes  a ‘value-directed’ 
benefit  dimension  and  thus  more  suitable  for  measuring  the  values  added  by  intangible  services  as  most 
learner  support  services.  He  proposed  three  types  of  benefit  measures  - performance-oriented,  value- 
oriented,  and  value-added  benefits,  and  it  is  believed  that  future  research  on  learner  support  can  benefit 
from  applying  his  multi-dimensional  framework. 

Secondly,  the  learner  support  models  based  on  correspondence  programs  and  their  underlying 
assumptions  need  to  be  tested  under  new  web-based  distance  education  programs  in  the  future  research. 
Large-scale,  text-based  distance  education  institutions  such  as  Open  University  in  the  UK  have  well- 
grounded  learner  support  systems  such  as  tutoring,  counseling,  and  advising,  and  have  served  as  a model 
for  learner  support  system  in  previous  research  (Tait,  1995;  Singh,  1988;  Sewart,  1993).  However,  dual 
mode  institutions  that  are  now  developing  web-based  distance  education  programs  are  facing  different 
challenges  in  establishing  and  standardizing  administrative  procedures  and  support  systems  to 
accommodate  new  technologies  and  diversified  learner  population. 

Thirdly,  future  research  needs  to  adopt  learner-centered  approach  in  designing  and  implementing 
learner  support  services  and  develop  ways  to  identify,  analyze,  synthesize,  and  assess  student  needs  and 
systematically  adapt  the  support  system  to  those  needs.  Thus  future  research  should  be  able  to  utilize 
various  methods  to  efficiently  communicate  with  distance  learners.  Research  on  student  attrition  is 
believed  to  provide  valuable  insights  to  understand  what  are  some  of  underlying  hindrances  or  barriers  in 
distance  learning  and  thus  help  future  research  on  learner  support  to  better  meet  the  learners’  needs. 
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Successful  performance  improvement  efforts  draw from  such  disciplines  as  psychology > and  systems  theory, 
and  from  the  fields  of  instructional  design  and  human  resource  development . Both  knowledge  management 
and  organizational  learning  are  valuable  additions  to  the  human  performance  technologist  *s  repertoire  for 
performance  analysis  and  intervention  selection.  In  this  paper,  I will  discuss  how  the  human  performance 
technologist  can  augment  human  performance  analysis  and  solution  planning  by  drawing  from  knowledge 
management  and  organizational  learning  literature. 

The  concept  of  organizations  as  learning  systems  emerged  at  the  beginning  of  the  twentieth 
century.  However;  it  was  not  until  the  1990s  that  we  witnessed  an  overwhelming  interest  in  the  impact  of 
knowledge  management  and  organizational  learning  on  organizational  performance. 

Current  research  on  knowledge  management  and  organizational  learning  has  dealt  primarily  with 
theory  and  model  building.  In  addition  to  the  empirical  research,  there  is  an  abundance  of  literature  based 
on  the  experiences  of  practitioners  and  facilitators  working  in  these  fields  (Davenport  & Prushak,  1998; 
Preskill  & Torres,  1999;  Hansen,  Nohria,  & Tierney,  1999).  The  existing  literature  represents  the  first  step 
in  a long-term  research  program.  The  second  research  phase  began  in  the  late  1990s.  It  involves  the 
development  and  validation  of  survey  instruments  to  diagnose  and  establish  learning  organizations;  and 
empirical  studies  that  test  models  and  describe  the  processes  and  outcomes  of  knowledge  management  and 
organizational  learning  on  organizational  performance  (O’Dell,  Grayson,  Jr.,  & Essaides,  1998;  Yeung, 
Ulrich,  Nason,  & Von  Glinow,  1 999). 

Knowledge  management  is  a conscious  strategy  of  getting  the  right  knowledge  to  the  right  people 
in  the  right  time,  and  helping  people  share  and  put  information  into  action  in  ways  that  strive  to  improve 
organizational  performance  (O’Dell,  Grayson,  Jr.,  & Essaides,  1998). 

The  focus  of  most  knowledge  management  literature  is  on  knowledge  generation,  codification  and 
transfer.  Knowledge  generation  encompasses  knowledge  acquired  by  an  organization  as  well  as  knowledge 
developed  within  it.  The  aim  of  codification  is  to  put  organizational  knowledge  hto  a form  that  is 
organized,  explicit,  portable,  easy  to  understand  and  accessible  to  those  who  need  it.  An  essential  element 
of  knowledge  management,  which  is  vital  to  the  organization’s  success,  is  to  develop  specific  strategies  to 
encourage  spontaneous,  unstructured  knowledge  transfer  (Davenport  & Prusak,  1998).  Nonoka  and 
Takeuchi  (1995)  defined  organizational  knowledge  creation  as  the  capability  of  a company  as  a whole  to 
create  new  knowledge,  disseminate  it  throughout  the  organization,  and  embody  it  in  products,  services  and 
systems.  Similarly,  DiBella  and  Nevis  (1998)  stated  that  organizational  learning  is  a cycle  of  three 
processes:  knowledge  creation  or  acquisition,  knowledge  dissemination,  and  knowledge  use. 

In  contrast,  O’Dell,  Grayson,  Jr.,  and  Essaides  (1998)  proposed  a seven-step  knowledge  process 
(create,  identify,  collect,  organize,  share,  adapt  and  use).  Yeung,  Ulrich,  Nason,  and  Von  Glinow  (1999) 
offered  a different  perspective  about  the  organization’s  capacity  to  learn.  They  proposed  that  an 
organization’s  fundamental  learning  capability  represents  its  capacity  to  generate  and  generalize  ideas  with 
impact  (change)  across  multiple  organizational  boundaries  (learning)  through  specific  management 
initiatives  and  practices  (capability). 
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Attention  has  also  been  given  to  the  sharing  of  tacit  knowledge  and  explicit  knowledge  at  the 
individual,  group  and  organizational  levels.  Both  tacit  and  explicit  knowledge  are  the  key  dynamics  of 
knowledge  creation  in  the  business  organization  (Nonoka  & Takeuchi,  1 995).  Davenport  and  Prusak  (1 998) 
explained  that  knowledge  that  is  explicit  can  be  embedded  in  procedures  or  presented  in  documents  and 
databases  and  transferred  with  reasonable  accuracy,  whereas  tacit  knowledge  transfer  generally  requires 
extensive  personal  contact  between  individuals  and  groups  within  the  organization.  The  process  of 
knowledge  sharing  is  necessary  if  the  organizational  interpretation  system  is  to  transcend  the  various 
interpretations  of  problems  and  solutions  at  the  individual  level  (Walsh  & Ungson,  1997). 

Literature  on  current  business  practices  indicates  that  consulting  businesses  employ  two  different 
knowledge  management  strategies  - codification  and  personalization.  The  codification  strategy  calls  for 
knowledge  to  be  carefully  codified  and  stored  in  databases  where  it  can  be  accessed  and  used  easily  by 
anyone  in  the  company.  With  the  personalization  strategy,  knowledge  is  closely  tied  to  the  person  who 
developed  it  and  is  shared  mainly  through  direct  person-to-person  contacts.  Companies  that  use  knowledge 
effectively  pursue  one  strategy  predominately  (80%)  and  use  the  second  strategy  (20%)  to  support  the  first 
(Hansen,  Noirha,  & Tierney,  1999). 

Organizational  learning  is  defined  as  an  adaptive  change  process  that  is  influenced  by  past 
practice,  focused  on  developing  or  modifying  routines,  and  supported  by  organizational  memory  (Nonoka 
& Takeuchi,  1 995).  The  essence  of  organizational  learning  is  members’  sharing  of  experiences  and  learning 
together  (Schwen,  Kalman,  Hara,  & Kisling,  1998). 

The  organizational  learning  literature  is  primarily  devoted  to  the  development  of  new  or  different 
organizational  structures  that  support  the  conditions  for  learning.  Managing  intellectual  capital  requires 
organizations  to  create  and  sustain  an  mvironment  where  employees  want  to  contribute  their  ideas, 
innovations,  and  analysis,  and  which  receives  them  willingly  (Horibe,  1999).  Tampoe  (1996)  states  that  a 
facilitative  environment  interacts  with  the  individuals  motivational  drive  and  competence  to  release 
motivational  energy.  This  motivated  energy  is  directed  into  professional  and  personal  achievement  by 
ensuring  that  individuals  have  a clear  sense  of  purpose  and  are  sustained  by  access  to  information  and  peer 
contacts. 

Conversely,  Weick  and  Westley  (1996)  argue  that  organizing  and  learning  are  antithetical 
processes,  which  means  the  phrase  organizational  learning  qualifies  as  an  oxymoron.  They  state  that  to 
learn  is  to  disorganize  and  increase  variety,  whereas  to  organize  is  to  forget  and  reduce  variety. 
Consequently,  organization  must  be  reduced  in  order  to  create  conditions  conducive  to  learning.  When 
organizations  are  allowed  to  exist  as  self-organizing  entities,  then  learning  and  knowledge  come  to  the 
surface  naturally,  because  survival  depends  on  it  (Cavaleri  & Fearon,  1996).  Lyles  and  Schwenk  (1997) 
hold  a similar  view  regarding  structures  that  support  organizational  learning.  In  tightly  linked  or  coupled 
knowledge  structures,  there  is  strong  consensus  among  organizational  members.  There  is  greater  rigidity  in 
the  sense  they  do  not  have  flexibility  in  responding  to  environmental  changes.  However,  loosely  coupled 
structures  incorporate  more  disagreement  and  alternative  interpretations.  Changes  can  be  made  easily  since 
there  is  more  flexibility  in  action  taking  and  strategies. 

Organizations  build  learning  capability  through  a variety  of  processes.  O’Dell,  Grayson,  Jr.}  and 
Essaides,  (1998)  promote  the  use  of  benchmarking  between  organizations  and  within  the  organization. 
Benchmarking  is  a process  of  systematically  finding  and  adapting  best  practices  in  order  to  improve 
performance.  A broader  view  of  how  organizations  approach  the  learning  process  is  supported  by  the 
research  conducted  by  Yeung,  Ulrich,  Nason,  and  Von  Glinow  (1999).  They  argue  that  there  are  four 
different  styles  of  organizational  learning:  experimentation,  competency  acquisition,  continuous 
improvement  and  benchmarking.  Competency  acquisition  and  continuous  improvement  are  the  most 
popular  learning  styles  based  on  survey  findings,  yet  experimentation  has  the  most  positive  effect  on 
business  performance. 

In  terms  of  best  practices,  many  successful  organizations  have  abandoned  hierarchical  structures, 
organizing  themselves  in  patterns  specifically  tailored  to  the  particular  way  their  professional  intellect 
creates  value  (Quinn,  Anderson,  & Finkelstein,  1 996).  In  inverted  organizations,  the  former  line  hierarchy 
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becomes  a support  structure.  Some  organizations  have  created  intellectual  (spider’s)  webs  in  which  people 
are  brought  together  quickly  to  solve  a problem  and  then  disbanded  just  as  quickly  when  the  job  is  done. 

An  applied  field  whose  aim  is  the  achievement  of  valued  human  performance  in  the  workplace  is 
human  performance  technology.  Human  performance  technologists  adopt  a systems  view  of  a performance 
gap.  They  systematically  analyze  both  gap  and  system,  and  design  cost-effective  and  efficient  interventions 
that  are  based  on  analysis  of  data,  scientific  knowledge  and  documented  precedents  in  order  to  close  the 
gap  in  the  most  desirable  manner  (Stolovitch  & Keeps,  1992).  Foshay  and  Moller  (1992)  describe  human 
performance  technology  as  an  applied  field  of  practice  that  is  structured  primarily  by  real-world  problems 
of  human  performance  (in  the  workplace).  It  draws  from  any  discipline  that  has  prescriptive  power  in 
solving  any  human  performance  problem.  It  also  may  draw  from  other  applied  fields  when  they  contribute 
technologies  of  use  in  solving  human  performance  problems. 

Performance  analysis  is  a process  for  defining  the  business  need  and  isolating  root  causes  of 
problems  within  existing  systems  or  for  identifying  opportunities  and  constraints  in  the  introduction  of  new 
structures,  systems  or  machines  (Brandenberg  & Binder,  1992).  Primary  interventions  used  by  human 
performance  technologists  include:  training,  job  aids,  feedback  systems,  employee  selection,  and 
organizational  technology  (Foshay  & Moller,  1992).  Schwen,  Kalman,  Hara,  & Kisling  (1998)  add  that  the 
human  performance  technology  analysis  process  involves  collecting  data  and  information  that  can  lead  to 
the  discovery  of  new  knowledge  and  make  tacit  knowledge  salient,  and  the  solutions  may  involve 
interventions  related  to  two  or  more  root  causes,  and  integrated  interventions  such  as  relevant  information 
data  bases,  coaching  and  mentoring,  and  modification  of  related  rewards  and  incentives. 

Some  common  themes  have  emerged  in  the  literature  with  respect  to  knowledge  management, 
organizational  learning  and  human  performance  technology.  Knowledge  management  involves  three  main 
processes:  generation,  codification  and  transfer  of  knowledge.  While  explicit  and  tacit  knowledge  are 
necessary  for  organizational  learning  to  occur;  it  is  recognized  that  because  tacit  knowledge  is  hard  to 
articulate  in  formal  language  it  is  also  more  dfficult  to  disseminate  and  transfer.  The  organizational 
learning  literature  indicates  organizations  that  have  developed  structures  and  strategies  that  nurture  and 
support  learning  have  experienced  improved  performance  despite  the  rapid  changes  facing  organizations. 
Human  performance  technology  relies  on  thorough  performance  analysis  to  identify  all  factors  contributing 
to  the  current  level  of  performance  and  to  propose  alternative  interventions  that  will  eliminate  the  cause  of 
the  performance  discrepancy. 

Another  trend  that  is  emerging  in  the  literature  involves  the  contribution  of  knowledge 
management  to  the  field  of  human  performance  technology.  Rossett  (1999)  outlines  that  knowledge 
management  perspectives  can  influence  analysis.  Analysts  would  provide  learners  with  a knowledge 
management  resource  that  provides  meaningfully  organized  data  elements.  To  develop  this  resource,  the 
analysts  must  capture  an  array  of  diverse  experiences  and  examples,  and  include  rich  commentary  that 
assures  a deeper  experience  for  users  when  they  choose  to  review  both  the  knowledge  element  and  people’s 
ideas  about  it.  Schwen,  Kalman,  Hara,  & Kisling  (1998)  state  that  the  knowledge  management  literature 
gives  linking  concepts  to  human  performance  analysis  (i.e.  making  tacit  knowledge  explicit,  identifying 
hidden  needs)  and  solution  planning  (i.e.  capturing  expert’s  knowledge,  mental  models). 

The  knowledge  management  literature  states  that  tacit  knowledge  is  hard  to  articulate  in  formal 
language  and  it  is  also  more  difficult  to  disseminate  and  transfer.  During  the  performance  analysis  process, 
the  human  performance  technologist  must  find  ways  to  make  this  tacit  knowledge  explicit.  In  addition  to 
the  standard  data-gathering  tools  (observation,  interviews,  surveys,  and  extant  data  analysis),  the  human 
performance  technologist  could  employ  critical  incident  analysis  to  draw  out  the  tacit  knowledge.  Critical 
incident  analysis  is  used  to  elicit  war  stories  by  asking  individuals  to  describe,  in  terms  of  behaviour,  what 
exactly  they  had  done  (correctly/incorrectly).  During  the  intervention  planning  phase,  the  human 
performance  technologist  must  take  into  consideration  that  explicit  knowledge  can  be  embedded  in 
procedures  or  presented  in  documents  and  databases  and  transferred  with  reasonable  accuracy,  whereas 
tacit  knowledge  transfer  generally  requires  extensive  personal  contact  between  individuals  and  groups 
within  the  organization.  Interventions  such  as  classroom  training,  policy  and  procedures  manuals,  data  base 
systems,  and  job  aids  are  generally  limited  to  the  transfer  of  explicit  knowledge.  On-the-job  training,  under 
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the  guidance  of  a coach  or  mentor,  should  be  considered  for  transferring  tacit  knowledge  that  is  closely  tied 
to  the  person  who  developed  it  ,or  which  is  shared  mainly  through  direct  person-to-person  contacts. 
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Both  knowledge  management  and  human  performance  technology  literature  focus  on 
competencies  (knowledge  and  skills)  required  for  individuals  to  perform  their  work  and  to  enable  the 
organization  to  maintain  its  competitive  advantage.  It  is  becoming  increasingly  important  for  organizations 
to  attract  and  retain  competent  individuals  with  exceptional  talent.  Human  performance  technologists 
should  examine  what  organizations  have  in  place,  or  should  have  in  place,  to  further  develop  each 
individual’s  knowledge  and  competencies,  and  to  support  each  individual’s  ability  to  contribute  to  the 
organization’s  objectives. 

Individuals  search  for  knowledge  because  they  expect  it  to  help  them  succeed  in  their 
work.  Individuals  learn  within  the  organization  when  they  acquire  knowledge  through  education, 
experience  or  experimentation.  Attention  should  also  be  given  by  human  performance 
technologists  to  identify  which  learning  style  (experimentation,  competency  acquisition, 
continuous  improvement,  or  benchmarking)  organizations  employ.  Since  organizations  learn  from 
both  direct  experience  and  the  experience  of  others,  the  human  performance  technologist  will  need 
to  look  at  intervention  designs  that  enable  the  system  and  culture  of  the  organization  to  retain  and 
transfer  knowledge  from  individuals.  In  this  way,  organizational  learning  will  be  embedded  in  the 
organization’s  routines,  technologies,  policies  and  procedures,  and  in  patterns  of  behaviour  that 
continue  to  exist  despite  turnover  of  individuals. 

The  literature  on  organizational  learning  indicates  that  managing  intellectual  capital 
requires  organizations  to  create  and  sustain  an  environment  where  employees  want  to  contribute 
their  ideas,  innovations,  and  analysis,  and  which  receives  them  willingly.  During  the  performance 
analysis  phase,  the  human  performance  technologist  should  study  the  organizational  structure  and 
job  requirements  to  determine  the  extent  to  which  the  organization  allows  for  the  existence  of 
naturally  occurring  learning  events.  In  order  to  plan  appropriate  solutions,  the  human  performance 
technologist  should  investigate  how  the  organization  enables  individuals  to  access  knowledge  that 
has  been  codified  and  stored  in  documents  and  databases,  and  how  it  fosters  personal  contact. 

Personal  contact  can  be  achieved  through  on-the-job  training  under  the  guidance  of  a coach  or 
mentor.  Alternatively,  organizations  could  form  action  learning  teams  or  employ  intellectual 
(spider’s)  webs  in  which  people  are  brought  together  quickly  to  solve  a problem  and  then 
disbanded  just  as  quickly  when  the  job  is  done. 

Future  endeavors  in  research  should  provide  empirical  evidence  of  the  value  of 
knowledge  management  and  organizational  learning  to  organizational  performance.  As  stated 
earlier  in  this  report,  the  study  of  knowledge  management  and  organizational  learning  is  currently 
moving  towards  broad-based,  empirical  research.  The  knowledge  gained  from  this  research  will 
enable  human  performance  technology  researchers  and  practitioners  to  implement  interventions 
based  on  tested  models,  and  proven  processes  and  outcomes  of  knowledge  management  and 
organizational  learning. 

There  is  also  a need  for  human  performance  technology  researchers  to  apply  paradigms 
for  research  that  will  be  both  effective  for  theory  development  and  appropriate  to  the  settings  of 
human  performance  technology  practice.  Human  performance  technology  by  its  nature  excludes 
use  of  experimental  paradigm  on  practical,  ethical  and  methodological  grounds.  However, 
researchers  will  find  descriptive  or  investigative  (case  studies)  most  useful  research  paradigms.  If 
researchers  and  practitioners  take  the  time  to  reflect  systematically  on  their  experience,  it  will  be 
possible  to  expand  the  empirical  base  of  the  field  (Foshay  & Moller,  1992). 
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Three  Approaches  to  Teaching  the  Same  Subject  at  Two 

Universities 


Ellen  Taricani 
Penn  State  University 


A collaborative  effort  was  established  between  two  major  universities  to  teach  a particular  subject.  The 
course  was  taught  3 different  ways:  full  collaboration,  half  collaboration,  and  a traditional  method.  This 
paper  will  discuss  the  methods  used  and  why  it  was  implemented  and  organized  in  this  manner. 

Background  to  the  Course 

In  the  fall  of  1997,  a senior  level  course  was  taught  as  an  interdisciplinary  class  for  learning  the 
processes  in  food  product  development.  The  course  implemented  many  new  teaching  methods  and 
technologies.  It  was  an  effort  between  two  departments:  food  science  and  agricultural  economics.  Students 
worked  in  teams  with  industry  mentors  who  assisted  in  elaborating  on  some  of  the  procedures  involved  in 
the  development  process  of  food  products.  It  was  intended  to  take  this  class  beyond  the  walls  of  Penn  State 
in  the  Fall  of  1998  and  include  some  other  locations.  An  opportunity  arose  to  link  up  with  students  in  a 
food  marketing  class  in  St.  Joseph's  University.  This  opportunity  included  finding  ways  to  share  resources 
and  for  the  students  to  get  involved  with  each  other.  Speakers  were  shared  and  some  collaborative  work 
was  done  during  the  1999  year  using  PictureTel.  The  classes  were  scheduled  for  the  same  time  of  day  to 
facilitate  cooperation. 

Fall  semester  of  2000,  it  was  decided  to  take  the  process  a step  further  and  truly  integrate  the 
teams  for  working  on  projects  and  assignments.  Teams  were  formed  to  work  together  o n the  course  project. 
There  were  three  different  versions  of  the  course.  One  course  is  totally  integrated.  The  second  version  will 
share  the  speakers  and  have  some  lectures  on  the  off  days.  And,  the  last  one  was  left  alone  to  be  taught 
traditionally  wi  th  lecture  and  few  outside  speakers  or  influences.  All  three  used  case  studies  to  assist  in  the 
learning  process.  In  all  three  cases  the  students  were  asked  to  create  concept  maps  and  take  a mini 
knowledge  test  to  be  used  in  the  evaluation  process  of  understanding  concepts  before  and  after  the  course. 


Aspects  of  the  Fully  Collaborative  Course 

There  are  several  distinct  features  of  the  fully  collaborative  food  product  development  course.  The  course 
was  originally  conceived  to  apply  principles  of  constructivist  theory.  In  this  approach,  prior  knowledge  and 
experience  is  the  springboard  for  useful,  personal  knowledge  construction.  "Constructivist  learning 
experiences  and  appropriate  classroom  practices  include  reflective  thinking  and  productivity;  authentic 
activities,  including  student  collaboration  and  consideration  of  multiple  perspectives,  and  student  access  to 
content  area  experts  who  can  model  domain-specific  skills."  (Grabe  & Grabe,  1998)  Course  organization 
was  based  on  the  idea  that  the  students  would  be  able  to  reflect  on  situations  and  environments  in  order  to 
make  application  towards  the  course  project. 

Jonassen  (1996)  also  has  much  to  say  about  constructivist  learning  environments.  "Constructivist 
environments  facilitate  learning  through  collaboration,  context,  and  construction  of  knowledge.  Through 
assimilation  and  accommodation,  individuals  use  many  elements  of  the  learning  context  and  relate  those 
elements  to  their  own  experiences  thus  creating  new  knowledge”.  Constructivism  does  not  always  produce 
predictable  learning  outcomes.  Instruction  should  foster  the  constructive  process  of  the  learner,  and  not 
attempt  to  closely  control  the  process  or  result.  The  process  of  instruction  and  role  of  instructor  should  be 
as  a guide  to  discovered  knowledge.  There  was  not  a specific  result  desired  in  the  course  rather  a process 
that  was  to  be  fine-tuned.  Contextual  situations  were  provided  with  the  case  studies  and  industry  speakers. 
The  use  of  case  studies  was  implemented  to  assist  in  developing  critical  thinking.  Students  were  expected 
to  apply  prior  knowledge  and  seek  out  new  knowledge  to  fill  in  the  gaps  that  may  rise  up  as  they  worked 
through  various  case  studies. 
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Objectives 


1 . To  comprehend  the  fundamental  principles,  generalizations  and  theories  of  product  development. 

2.  To  understand  the  specific  skills,  competencies  and  points  of  view  needed  by  product  development 
professionals. 

3 . To  appreciate  the  interdependencies  of  finance,  formulation,  marketing,  packaging,  process 
engineering,  production  and  quality  assurance  in  the  development  of  food  products. 

4.  To  better  understand  how  food  company  managers  gain  knowledge  about  the  process  for 
developing  food  products. 

5.  To  develop  skills  in  analyzing  ill-defined  opportunities  and  developing  strategies  to  resolve  them. 

6.  To  develop  the  competencies  of  effective  communication  and  negotiation  skills  when  working 
within  groups/teams. 

7.  To  develop  the  ability  to  be  a self-regulated  learner  who  can  engage  in  both  constructive  and 
critical  self-  and  peer-assessment. 

8.  To  enhance  the  ability  to  use  contemporary  communications  and  information  technology.  To 
enhance  oral  and  written  communications  skills. 

Use  of  Case  Studies 

Wilson  (1996)  defines  a constructivist  learning  environment  as:  "a  place  where  learners  may  work 
together  and  support  each  other  as  they  use  a variety  of  tools  and  information  resources  in  their  guided 
pursuit  of  learning  goals  and  problem-solving  activities”  (p.5).  He  emphasizes  learning  environments  as 
opposed  to  'instructional'  environments  in  order  to  promote  "a  more  flexible  idea  of  learning”,  one  which 
emphasizes  "meaningful,  authentic  activities  that  help  the  learner  to  construct  understandings  and  develop 
skills  relevant  to  problem  solving”  (p.3). 

Case  studies  have  the  capacity  to  engage  students  in  the  thinking  process.  Case-based  reasoning  is 
a cyclic  and  integrated  process  of  solving  a problem,  learning  from  this  experience,  and  solving  new 
problems.  The  roots  of  case-based  reasoning  can  be  found  in  the  works  of  Roger  Schank  on  dynamic 
memory  and  the  central  role  that  a reminding  of  earlier  situations  (episodes,  cases)  and  situation  patterns 
(scripts,  MOPs)  has  in  problem  solving  and  learning  (Schank,  82).  Each  section  of  the  case  takes  the 
student  into  new  areas  of  cognitive  exploration.  Presenting  the  cases  on  the  web  as  opposed  to  a written 
format  provides  the  students  with  opportunities  to  explore  and  select  particular  information.  Information 
that  is  presented  without  any  direct  and  interactive  activity  will  often  pass  right  through  their  head  without 
any  processing  or  application.  They  are  not  likely  to  change  the  way  they  view  a particular  situation  as  a 
result  of  merely  hearing  the  information.  Due  to  the  nature  of  most  web  pages,  students'  minds  will  race 
past  topics  in  rapid  succession  with  little  time  or  opportunity  to  think,  reflect  or  develop  meaning.  Their 
cognitive  skills  are  only  apparent  on  a scattered  layer  of  thinking.  They  are  forced  to  adopt  a surface 
approach  to  their  learning.  Students  are  typically  in  a survival  mode  to  read,  memorize  and  reiterate  the 
information  in  some  sort  of  examination  process. 

Some  of  the  problems  of  student  understanding  and  learning  can  be  enhanced  through  the  use  of 
alternative  teaching  approaches.  This  paper  discusses  a model  used  to  promote  critical  thinking  on  the  web 
using  case  studies.  Aside  from  the  basic  issue  of  facilitating  effective  learning,  there  is  a need  to  develop  in 
students  a method  for  problem-solving  that  will  be  useful  in  "real  world"  examples. 

Interactive  Aspects 

Interactivity  in  the  course  invites  the  students  to  participate  and  become  part  of  what  is  going  on 
with  the  industry  and  the  people  influencing  it.  The  cases  were  written  for  this  class  with  the  goals  of 
instruction  in  mind.  Each  case  had  its  own  particular  items  that  provided  twists  in  the  development 
process.  Some  gave  the  students  very  difficult  decision  points  that  required  outside  sources  to  gain  more 
insights  on  the  subjects.  Students  worked  both  individually,  and,  in  groups  to  share  ideas  and  solve  the 
problems.  Experts  in  the  field  were  brought  in  through  face-to-face  presentations  or  over  PictureTel 
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sessions.  Email  and  phone  calls  were  also  used  to  give  the  students  access  to  probe  and  investigate  some  of 
the  key  points  in  the  case  as  well  as  questions  related  to  their  projects.  Cases  were  shared  with  a discussion 
from  both  the  instructor  and  student  view. 


Stimulating  Learning 

A great  deal  is  known  about  factors  that  affect  learning  and  in  particular  factors  that  facilitate  a deep 
approach  to  learning.  There  are  some  specific  methods  that  can  be  used  to  motivate  students.  Case  studies 
provide  the  opportunity  for  critical  thinking  since  they  only  provide  part  of  the  information  needed  to 
complete  the  exercises.  Students  will: 

• formulate  questions  that  relate  to  the  current  case 

• construct  knowledge  while  they  are  engaged  in  problem  solving 

• build  on  and  diffuse  what  they  already  know 

• have  the  opportunity  to  discuss,  explain,  write  and  reflect  on  the  new  knowledge 

• experience  some  control  over  what,  when  and  how  they  are  learning 

• have  an  opportunity  to  consider  new  possibilities 

• receive  feedback  regarding  their  learning 

Learners  actively  take  knowledge,  connect  it  to  previously  assimilated  knowledge  and  make  it  theirs  by 
constructing  their  own  interpretation.  Students  will  read  a case  with  their  own  experiences  and  a cognitive 
structure  based  on  those  experiences.  These  structures  can  be  valid,  invalid  or  incomplete.  The  learner  will 
reformulate  his/her  existing  structures  only  if  new  information  or  experiences  are  connected  to  knowledge 
already  in  memory.  The  learner  must  actively  construct  new  information  onto  existing  mental  framework 
for  meaningful  learning  and  problem  solving  to  occur. 

Selection  of  the  Tools 

Select  appropriate  online  technologies  such  as  video  and  audio  conferencing,  computer-mediated 
conferencing,  Internet  access,  bulletin  boards  and  news  groups,  and  e-mail  to  support  teaching  and  learning 
strategies.  A short  discussion  of  each  tool  and  reasons  to  use  them  will  follow. 


TOOL 

BENEFIT 

Video  and  audio  conferencing 

Provides  visual  and  audio  cues 

Computer-mediated  conferencing 

Initiate  discussions 

Internet  access 

A wealth  of  resources 

Bulletin  boards  and  news  groups 

Others  interested  in  similar  discussions 

E-mail 

Direct  contact  with  teachers,  students,  and  speakers 

Tools  to  promote  higher-order  thinking  should  be  used  in  this  constructivist  process.  It  is  not  only 
essential  to  place  the  students  in  a rich  environment  full  of  new  technologies,  but  to  provide  a means  and 
reason  for  using  the  tools.  PictureTel  was  selected  as  a video  conferencing  method  since  both  universities 
had  access  to  it.  It  was  used  at  least  once  a week  in  order  to  share  speakers  and  provide  access  for  group 
communication.  Courselnfo  was  used  as  a course  management  system  with  advanced  architecture  that 
allowed  for  Web-based  integration  and  administrative  systems.  This  provided  the  framework  for 
communication  with  the  students  and  amongst  the  students.  Students  used  the  online  system  to  have 
discussions  and  share  information  with  other  team  members.  Faculty  used  the  system  to  post  assignments 
and  information. 

Evaluation  of  Student  Performance 

Students  were  evaluated  and  the  course  grade  assigned  based  on  the  following: 
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Individual  Performance 

Team  Performance 

Final  Examination 

15% 

Progress  Report: 
Oral 

15% 

Peer  Evaluation  of 
Collaboration  on  Team 

20% 

Final  Report: 
Written 

20% 

Participation 

15% 

Final  Report:  Oral 

15% 

TOTAL 

50% 

TOTAL 

50% 

The  Final  Examination  (15%)  is  a description  of  the  product  development  process  for  a specific  food 
product  opportunity.  Peer  Evaluation  of  Collaboration  on  Team  (20%)  is  accomplished  by  asking  students 
to  rate  their  team  members  on  eight  different  parameters  and  then  to  offer  an  overall  weight  by  assigning 
the  portion  of  a SI  0,000  bonus  to  be  distributed  to  each  team  member.  This  evaluation  will  be  done  twice 
during  the  semester  and  at  the  end  of  the  semester. 

The  Progress  Report  (15%)  is  a team-based  oral  presentation  simulating  the  product  development 
team’s  presentation  of  its  plans  to  the  firm’s  upper  management  for  a ‘go-no  go’  decision.  The  Final 
Report  (35%)  is  their  team’s  presentation  of  the  food  product  innovation  to  retail  buyers. 


Aspects  of  the  Partially  Collaborative  Course 

The  class  in  this  partially  collaborative  course  was  taught  at  St.  Joseph's  University.  It  consisted 
of  a combination  of  lecture,  presentations,  discussion  and  exercises.  Guest  presentations  from  food 
industry  professionals  were  provided.  The  major  emphasis  of  the  course  was  on  the  new  product  planning 
process  and  to  that  end,  each  student  was  assigned  to  a student  team.  Each  team  took  on  the  identity  of  a 
food  processing  company  and  was  challenged  to  identify  a new  product  opportunity,  conceive  a new 
product,  and  develop  a launch  program  to  support  introduction  of  this  new  product  to  the  supermarket 
trade.  Both  a written  paper  and  a formal  oral  presentation  were  required  of  each  student  team.  The  final 
presentations  were  made  to  a group  of  retail  category  managers.  The  final  presentations  were  held  during 
the  scheduled  time  for  the  final  exam. 

Course  Objectives 

1 . To  develop  skills  necessary  to  be  a contributing  member  of  a team. 

2.  To  enhance  oral  and  written  communications  skills. 

3.  To  develop  skills  in  analyzing  ill-defined  problems  and  developing  strategies  to  resolve  them. 

4.  To  better  understand  how  food  company  managers  gain  knowledge  about  the  process  for  developing 
food  products. 

5.  To  comprehend  the  fundamental  principles,  generalizations  and  theories  of  product  development. 

6.  To  understand  the  specific  skills,  competencies  and  points  of  view  needed  by  product  development 
professionals. 

7.  To  appreciate  the  interdependencies  of  finance,  formulation,  marketing,  packaging,  production  and 
quality  assurance  in  the  development  of  food  products. 

Class  Format 

The  class  format  consists  of  a combination  of  lecture,  presentations,  discussion  and  exercises.  The 
instructor  will  endeavor  to  provide  guest  presentations  from  food  industry  professional  s on  at  least  a couple 
of  occasions. 

Assignments  and  Grading  Policy 
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This  course  is  primarily  a team-based  course.  Each  student  had  one  major  individual  task  - a midterm 
examination.  The  bulk  of  the  grade  was  determined  by  the  quality  and  completeness  of  the  work  on  the 
semester  project.  The  grade  was  determined  as  follows: 

Midterm  examination 40% 

Individual  grade  on  NPD  project  50% 

The  project  grade  will  be  divided  into  two  components,  the  preliminary 
v presentation  to  the  management  committee  (1/3)  and  the  final 

presentation  (1/3)  and  comprehensive  team  paper  (1/3.) 

Contadina  case  (special  project) 10%* 

*If  you  opt  NOT  to  participate  in  the  Contadina  special  project,  you  will 

need  to  complete  a short  research  paper  on  a topic  assigned  by  the  instructor. 

Note:  The  individual  grade  for  team  assignments  will  be  the  team  grade  multiplied  by  an  index 
number  which  reflects  the  quantity  and  quality  of  your  contribution  to  the  team  effort. 

The  project  consists  of  several  parts,  including  a (1)  company  review  and  assessment,  (2)  category 
assessment,  (3)  concept  development  and  screening,  (4)  product  development  and  testing  and  (5) 
introductory  marketing  program  and  launch  plan.  Class  time  will  be  used  to  discuss  more  specifics  about 
the  project  and  details  on  the  midterm  and  the  project.  The  paper  is  due  at  the  time  of  the  final 
presentation. 


Aspects  of  the  Traditional  Course 

The  third  course  was  taught  in  a more  traditional  mode.  There  were  lectures,  tests  and  a project. 

This  class  also  participated  in  some  of  the  PictureTel  sessions  with  the  industry  speakers. 

Course  Objectives 

1 . To  comprehend  and  apply  the  fundamental  principles,  generalizations  and  theories  of  product 
development. 

2.  To  understand  and  apply  the  skills,  competencies  and  viewpoints  needed  by  product  development 
professionals. 

3.  To  appreciate  the  interdependencies  of  R&D,  manufacturing,  finance,  marketing  and  sales  in  the 
development  of  successful  food  products. 

4.  To  develop  skills  in  analyzing  and  taking  advantage  of  ill-defined  opportunities. 

5.  To  understand  the  risks  and  rewards  associated  with  new  food  products. 

6.  To  enhance  personal  skills:  team  work;  oral  presentation;  written  presentation;  selling. 


Grade  Components  and  Weights 

Semester  Project:  Written: 

40% 

Oral: 

20% 

Special  Semester  Project: 

10% 

Class  Participation: 

15% 

Midterm  Examination: 

15% 

Semester  Project 

The  course  has  three  major  emphases:  developing  a new  food  product;  selling  the  concept  to  top 
management;  selling  the  product  to  the  retail  supermarket  trade.  Each  student  was  assigned  to  a team.  Each 
team  will  select  and  assume  the  identity  of  a food  manufacturing  company  and  will:  identify  a new  product 
opportunity;  conceive  a new  product;  and  develop  a launch  program  to  introduce  the  new  product  to 
consumers  in  coordination  with  the  supermarket  distribution  channel. 
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Each  team  submitted  a written  paper  of  20  pages  or  less,  excluding  appendices,  detailing  its  new 
product  development  and  launch  programs.  The  paper  was  due  on  December  6.  An  in-process  draft  was 
due  on  November  21 . 

Each  team  prepared  a sell-in  presentation  to  be  made  to  a panel  of  grocery  category  managers  in 
December,  exact  date  and  place  to  be  determined. 


The  Study 

This  study  is  primarily  looking  at  the  differences  in  learning  using  different  teaching  methods.  The 
evaluation  of  the  students  was  essential  to  measure  those  differences.  There  was  a pre-test  and  a post-test 
to  assist  in  analysis.  Many  methods  were  implemented  that  will  be  discussed. 

Conventional  Teaching 

There  are  typical  elements  found  in  conventional  teaching  practices.  The  curriculum  content  is  usually 
fixed  and  presented  in  linear  and  sequential  ways.  A path  of  learning  is  established  and  sequentially  used 
for  the  course.  Mastery  of  existing  knowledge  and  concepts  is  sought  before  students  are  led  to  the  next 
set.  Typical  activities  involve  learning  tasks  that  are  segmented  and  fragmented  to  make  them  more  easily 
achieved.  These  can  include  reading  and  lectures  relevant  to  each  particular  subject.  The  activities  tend  to 
lack  any  real  life  context  and  are  usually  presented  in  abstract  forms  bearing  little  relevance  to  settings 
beyond  the  classroom.  In  most  cases,  the  teacher  plays  the  role  of  the  expert  delivering  knowledge  to  the 
learners.  Learners  act  in  passive  modes  working  individually  to  complete  the  set  tasks.  Assessment  of 
learning  is  done  through  pencil  and  paper  tests  measuring  competency  in  the  various  elements  of  the 
curriculum. 

Learners  have  frequently  been  found  to  be  incapable  of  applying  and  transferring  the  learning  to 
practical  settings.  Learning  has  been  found  to  be  temporary  and  short-lived. 

Active  Learning  and  Teaching  Methods 

For  some  time  now,  educational  researchers,  classroom  teachers  and  curriculum  developers  have 
been  exploring  ways  to  increase  the  effectiveness  of  teaching  programs  and  in  particular,  classroom 
learning.  The  learning  theories  have  always  suggested  that  what  is  needed  is  more  active  involvement  of 
the  learners  in  the  learning  process.  Theories  of  learning  have  been  developed  which  explain  the  way  in 
which  learning  is  achieved  through  knowledge  construction.  The  integral  role  of  communication  between 
learners  has  been  explored  and  the  value  of  collaboration  and  co-construction  of  knowledge  developed.  At 
the  same  time,  curriculum  developments  have  moved  from  descriptions  of  the  content  to  be  learned  to 
environments  where  outcomes  of  learning  have  been  made  discrete.  The  role  of  assessment  has  been 
recognized  and  given  a more  fundamental  place  in  the  learning  process.  The  sum  of  these  developments 
suggests  a changed  direction  for  educational  planning. 

Changed  Roles  for  Learners 

The  first  thing  often  observed  about  learners  in  an  active  learning  environment  is  the  degree  of 
self-regulation  and  self-determination.  Students  are  expected  to  search  for  meaning  throughout  the  course 
through  resources  provided.  Formal  structures  are  removed  and  students  are  free  to  make  their  own 
connections  and  pathways.  Some  of  these  include: 

• Freedom  of  Information 

Information  is  provided  in  an  open  setting  such  as  the  web,  speakers,  case  studies  and  the  course 
faculty.  A book  is  also  used  in  this  setting  to  provide  examples  and  more  detailed  explanations.  Much  of 
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the  information  that  was  gathered  by  the  students  came  from  prior  experience,  discussions,  web  pages, 
speakers,  interactions  with  the  faculty  and  cross-institutional  dialog. 

• Active  Learners 

Students  are  encouraged  to  collaborate  and  work  together.  The  environment  is  usually  one  of  a shared 
learning  space  with  learners  attentive  and  receptive  to  others  in  the  class.  Students  are  asked  to  solve 
problems  and  ask  questions.  They  are  expected  to  work  with  the  other  students  in  their  group  to 
accomplish  tasks. 

• "Real-life"  Activities 

Activities  in  class  encourage  and  support  such  strategies  as  problem-based  learning,  case-based 
learning  and  presentations  from  those  in  the  industry.  The  concept  of  a classroom  as  a place  of  learning  is 
expanded  as  the  classroom  loses  it  boundaries.  Each  case  study  was  written  with  the  specific  goals  of  the 
course  in  mind.  It  was  important  to  provide  examples  of  the  various  stages  in  the  food  product 
development  cycle  so  that  the  students  could  make  application  toward  their  final  project. 


Changed  Roles  for  the  Teachers 

Teachers  in  active  learning  environments  differ  in  terms  of  their  roles  and  responsibilities.  The 
differences  appear  in  how  they  interact  with  their  learners  and  how  they  manage  and  implement  their 
learning  settings.  Some  of  these  roles  include: 

• The  Coach 

Their  role  is  that  of  a guide  or  a coach.  They  provide  the  learners  with  access  to  a variety  of 
independent  learning  experiences.  There  are  minimum  times  that  there  is  a need  for  lecturing  or  other  forms 
of  teacher-directedness  in  these  settings.  The  most  active  person  in  the  environment  is  the  learner  and  often 
the  teacher  is  a spectator  of  learning  shouting  advice  from  the  sidelines. 

• Instructional/Leaming  Designer 

The  teacher  will  play  a vital  role  in  designing  the  learning  activities  and  developing  creative  ways  to 
involve  the  students.  Instead  of  only  considering  what  is  being  taught,  the  teacher  has  to  be  thinking  of 
how  it  is  taught  and  what  the  possible  outcomes  will  be. 


• Assessment 

The  move  to  an  outcomes-oriented  approach  carries  with  it  changes  to  assessment  strategies.  These 
strategies  reflect  how  the  learning  is  to  be  used.  Some  of  these  approaches  include  teamwork  strategies, 
case  study  analysis,  projects,  presentations,  and  summary  papers. 


Summary 

The  three  courses  will  be  used  to  make  comparisons  in  the  teaching  methods  to  find  differences  in 
outcomes  for  the  students.  Each  course  contains  components  of  constructivism  in  the  approach  to  teaching 
especially  regarding  the  use  of  case  studies.  The  major  differences  are  the  methods  of  learning  and  the 
involvement  of  the  students  in  the  learning  process.  Students  will  be  surveyed  at  the  end  with  another 
concept  map  and  mini  test  to  compare  the  results  from  the  beginning  of  the  class  to  the  end.  The 
knowledge  presented  was  similar,  but  the  methods  used  were  extremely  different.  Some  of  the  questions 
to  be  answered  regard  the  depth  and  application  of  the  concepts  that  are  obtained. 
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Since  the  start  of  the  early  90’s,  an  increasing  number  of  people  are  interested  in  supporting  the 
complex  tasks  of  the  curriculum  development  process  with  computer-based  tools.  ‘Curriculum 
development’  refers  to  an  intentional  process  or  activity  directed  at  (re)  designing,  developing  and 
implementing  curricular  interventions  in  schools,  colleges,  or  corporate  education.  The  term  ‘curricular 
intervention’  serves  as  a common  denominator  for  curricular  products,  programs,  materials  (varying  from 
teacher  and  student/trainee  materials),  procedures,  scenarios,  processes,  and  the  like.  A key  issue  and  major 
challenge  in  curriculum  development  is  how  curricular  interventions  should  be  developed  in  order  to 
achieve  a satisfying  balance  between  the  ideals  of  a curriculum  change  and  their  realization  in  practice. 
This  paper  provides  background  information  on  the  roots  of  computer-based  tools  for  curriculum 
developers,  provides  an  overview  of  those  tools  currently  available  in  the  USA  and  abroad  (especially 
Australia  and  The  Netherlands)  and  describes  probable  future  trends. 


1.  Roots  of  Computer-Based  Tools  for  Curriculum  Developers 


Many  designers  make  use  of  tools  or  job  aids  providing  design  support  in  their  daily  work.  A job 
aid  is  a collection  of  several  kinds  of  conceptual  or  procedural  information  (for  instance:  glossaries  of 
terms,  guidelines,  decision  tables,  checklists)  that  supports  work.  Over  the  years,  many  of  these  job  aids 
have  been  combined  into  handbooks  for  instructional  designers.  According  to  Rossett  and  Gautier- Downes 
(1991),  job  aids  may  have  major  advantages  for  their  users,  such  as: 

• they  are  available  at  the  moment  individuals  feel  a need  for  them; 

• they  increase  the  chance  that  an  individual  has  up-to-date  information  to  perform  a task, 
especially  in  case  of  a very  complex  and  infrequently  performed  activities; 

• they  prompt  individuals  through  difficult  processes  and  decisions. 

Computer-based  technologies  have  not  only  influenced  the  domain  of  job  aids.  They  have  also 
impacted  other  types  of  external  support,  such  as  communication  and  training.  Today's  computer  and 
networking  facilities  can  even  integrate  these  types  of  performance  support.  Instead  of  separately  providing 
different  ways  of  support  to  individuals,  an  electronic  performance  support  system  (EPSS)  provides 
integrated  information,  advice  and  learning  opportunities  to  improve  user  performance  (Gery,  1991; 
Raybould,  1995).  EPSSs  are  giwn  many  names,  such  as  performance  support  tools  (Carr,  1992), 
(integrated)  performance  support  systems  (Geber,  1991),  embedded  performance  support  systems 
(McGraw,  1995).  But  regardless  of  the  terminology  used,  they  all  refer  to  a computer-based  system  which 
provides  integrated  support  in  the  format  of  any  or  all  of  the  following:  job  aids  (including  conceptual  and 
procedural  information  and  advice),  communication  aids  and  learning  opportunities  to  improve  user 
performance. 

With  the  increase  of  the  number  of  computers  used  at  work,  a growing  number  of  computer-based 
tools  for  designers  and  developers  in  education  and  training  have  been  developed  at  various  places  around 
the  world.  More  efficient  development  processes,  more  effective  learning  programs  and  increasingly 
competent  designers  are  all  potential  benefits  that  make  these  tools  attractive  to  many  designers  and  their 
managers.  In  addition  to  these  assumed  advantages,  some  criticism  may  also  be  found  in  literature.  Firstly, 
the  potential  supportive  role  of  these  tools  should  be  carefully  judged.  For  instance,  as  the  consultation  of 
an  EPSS  is  usually  largely  self-directed,  certain  capacities  of  the  individuals  who  use  the  EPSS  are 
required:  they  need  to  know  what  they  do  not  know;  value  a high  degree  of  control  and  be  able  to  evaluate 
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the  quality  of  the  information,  to  name  just  a few  characteristics.  Not  every  individual  possesses  these 
cognitive  as  well  as  affective  characteristics.  Moreover,  Clark  (1992)  suggests  that  in  many  dynamic  work 
environments,  individuals  do  not  have  the  time  to  look  for  information  in  the  job  aids  or  leam  from  the 
CBT  component  of  an  EPSS.  Developers  of  EPSSs  and  organizations  who  consider  using  these  tools 
should  take  these  potential  problems  into  account. 


2.  Overview  of  Available  Tools 


In  order  to  get  an  overview  we  examined  the  following  available  tools:  GAIDA,  QIPP  EPSS, 
PLATO,  MediaPlant,  SimQuest,  CASCADE-SEA,  TeleTOP  DST,  Mercator,  IDXelerator,  AGD  and  GTE. 
In  box  1 each  tool  is  briefly  introduced.  For  more  information  please  refer  to  the  references  of  each  tool. 


GAIDA:  Guided  Approach  to  Instructional  Design  Advising  (GAIDA)  offers  on-line  elaborated 
guidance  for  the  application  of  Gagne’s  nine  events  of  instruction  (e.g.  Gagne,  Briggs  & Wager,  1 992)  to 
the  design  of  interactive  courseware  and  other  instructional  materials.  GAIDA  was  developed  for  novice 
instructional  developers  at  the  Air  Force  Research  Laboratory  (Gettman,  McNelly  & Muraida,  1999). 

QIPP  EPSS:  This  tool  supports  the  application  of  a new  development  methodology  (called 
Quality  Information  Products  Process)  for  designing  technical  documentation  at  NCR.  The  system  specifies 
phases  and  work  activities  of  the  instructional  design  process  and  provides  job  aids  for  each  activity. 
Technical  writers  and  instructional  designers  of  NCR  belong  to  its  main  target  group  (Jury  & Reeves, 
1999). 

Plato  Courseware  Development  Environment:  Plato  is  an  authoring  tool  to  support  the  design 
and  development  of  courseware  including  tutorials,  simulations  and  constructivist  learning  environments. 
The  systems  can  be  used  by  non-programmers  to  author  instructional  activities  by  customizing  objects  that 
are  copied  from  a library  and  assembled  into  completed  multimedia  components.  For  each  phase  of  the 
process  there  are  job  aids  accessible  to  all  members  of  the  design  team  (Preese  & Foshay,  1999). 

MediaPlant:  This  is  a development  environment  that  facilitates  the  production  of  complex  cross 
platform  learning  environments.  The  development  program  is  used  to  construct  and  test  the  learning 
environment,  which  is  then  distributed  with  the  runtime  program  (Wright,  Harper  & Hedberg,  1999). 

SimQuest:  This  is  an  authoring  environment  for  creating  learning  environments  that  combines 
simulations  with  instructional  support  that  helps  learners  in  the  process  of  discovery  learning.  An  author 
(teacher)  creates  a learning  environment  by  adapting  building  blocks  selected  from  a library.  The  author 
gets  support  from  an  on-line  help  system,  a wizard  and  an  advice  tool  (de  Jong,  Limbach,  Gellevij,  Kuyper, 
Pieters  & van  Joolingen,  1999). 

CASCADE-SEA:  CASCADE-SEA  (Computer  Assisted  Curriculum  Analysis,  Design  and 
Evaluation  for  Science  Education  in  Africa)  aims  to  support  curriculum  development  within  the  context  of 
secondary  level  science  and  mathematics  education  in  sub-Saharan  Africa.  One  of  its  components  is  called 
“lesson  builder”.  This  component  has  been  designed  to  help  teachers  make  paper-based  exemplary  lesson 
materials.  Based  on  input  of  the  user,  Lesson  Builder’s  prompt  the  program  to  generate  a draft  (McKenney, 
1999). 

TeleTOP  DST:  The  TeleTOP  Decision  Support  Tool  is  a WWW-based  environment  that  helps 
instructors  become  aware  of  technical  possibilities  for  their  courses  and  helps  them  to  see  how  these  could 
be  integrated  in  an  educationally  useful  way  (Collis  & de  Boer,  1999). 

Mercator:  This  system  supports  the  design,  production  and  delivery  of  course  materials.  On  the 
one  hand  it  helps  to  design  and  produce  the  material,  and  on  the  other  hand  it  helps  students  to  select 
specific  materials  and  supports  the  actual  delivery  in  a printed  and/or  electronic  mode  (Valcke,  Kirschner  & 
Bos;  1999). 

IDXelerator:  This  authoring  system  automatically  generates  the  instructional  interactions  required 
for  the  student  to  acquire  a specific  kind  of  knowledge  or  skill.  The  system  has  an  author  view  (that 
supports  the  author)  and  a student  view  for  delivery  of  the  instruction  (Merrill  & Thompson,  1999). 
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AGD:  Atelier  de  Genie  Didactique  provides  a pedagogical  design  assistant  to  content  experts  in 
companies  and  university  teachers  in  preparing  lessons  for  distance  learning  settings  (Paquette,  Aubin  & 
Crevier,  1994;  Spector;  1999). 

GTE:  The  Generic  Tutoring  Environment  is  focused  on  providing  support  for  designing  intelligent 
tutoring  systems.  The  primary  task  of  such  systems  is  to  integrate  instructional  knowledge  in  the  system  in 
a way  that  allows  the  system  to  adapt  to  learners  just  as  expert  teachers  do  (van  Marcke,  1 998;  Spector, 

1999). 

Box  1 . Brief  description  of  computer-based  tools  to  support  curriculum  developers 

From  the  short  overview  in  box  1 it  becomes  clear  that  available  computer-based  support  tools  for 
developers  in  the  field  of  training  and  education  can  be  classified  in  many  ways.  To  analyze  the  tools  in 
more  detail  we  used  a framework  with  the  following  set  of  attributes: 

A.  Type  of  output: 

• Curriculum  level  (few  lessons,  product,  course,  collection  of  courses) 

• Characteristics  of  results  (target  group,  form,  extensiveness) 

B.  Purpose  and  evidence  of  benefits: 

• Purpose  (transfer  of  knowledge  and  skills,  improved  task  performance,  organizational 
learning 

• Evidence  of  claimed  benefits  (validity,  practicality,  effectiveness) 

C.  Type  of  development  process  supported  and  any  underlying  theory: 

• Paradigm  for  engaging  in  education  and  training  benefit  (instrumental,  communicative, 
pragmatic,  artistic) 

• Elements  of  systematic  approach  (analysis,  design,  development,  implementation, 
evaluation) 

• Underlying  teaching/leaming  theory  (behaviorism,  cognitivism,  constructivism) 

D.  Task  support: 

• Types  of  support  (communication  aids,  job  aids,  training  aids) 

• Adaptability  of  support  (outside  the  tool,  inside  the  tool,  inside  networked  tool,  closed) 

E.  Intended  user  group: 

• Expertise  of  user  group  (professional  designer  (ISD),  subject  matter  expert,  teacher, 
learner) 

• Scope  of  intended  user  group  (various  organizations,  specific  organization) 

• Computer  experience  (low,  high) 

The  framework  was  used  to  examine  the  tools  for  developers  mentioned  in  box  1 (see  Table  1). 
The  framework  should  be  judged  solely  on  its  utility  as  a schema  for  examining  and  selecting  from  among 
tools  and  is  not  intended  to  be  a scientifically  valid  taxonomy.  Moreover,  it  should  be  noted  that  the 
analysis  is  based  on  limited  information  provided  by  the  developers  of  the  tools  and  is  not  based  on 
personal  experience  with  most  of  the  tools.  For  an  individual  who  wants  to  select  a tool  based  on  the 
information  in  the  framework,  technical  issues  such  as  needed  operating  system,  software;  and  hardware 
would  be  critical  to  consider  as  well.  The  same  is  true  for  other  i ssues  such  as  time  needed  to  learn  the  tool; 
time  needed  until  a user  starts  to  be  productive  with  the  tool;  the  costs  of  a tool;  and  its  general  availability. 
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Here  we  provide  a brief  interpretation  of  the  analysis. 


A.  Type  of  Output 

In  general,  the  use  of  most  of  the  tools  results  in  instructional  products  or  courses  that  are 
computer-based  and  have  learners  as  their  main  target  group.  Although  it  might  be  argued  that  many,  if  not 
all,  of  the  tools  could  be  used  for  creating  many  different  forms  of  instruction,  they  do  appear  to  lend 
themselves  best  to  one  or  only  a few  forms  of  output.  None  of  the  systems  seem  to  focus  primarily  on  the 
development  of  interrelated  collections  of  courses.  Only  a few  of  them  may  lead  to  paper-based  materials 
(GAIDA,  CASCADE- SEA  and  Mercator),  and/or  web- based  materials  (PLATO,  DST,  and  Mercator)  and 
only  three  tools  support  the  development  of  teacher-based  materials  (CASCADE-SEA,  DST,  and  AGD). 


B.  Purpose  of  the  Tool 

Generally  speaking,  all  tools  are  designed  with  the  expectation  of  improving  the  performance  of 
developers  of  training  and  education.  In  describing  the  tools,  some  authors  anticipate  that  their  tool  will 
lead  to  a better  transfer  of  knowledge  and  skills  to  the  actual  task  performance  because  it  makes  the 
rationale  of  the  tool  (and  thus  that  of  the  design  process)  explicit  (cf.  QIPP,  SimQuest,  CASCADE-SEA, 
and  AGD).  Others  expect  that  the  use  of  their  tool  will  lead  to  organizational  learning,  since  the  tool 
invites  users  to  make  newly  acquired  information  available  to  their  whole  organization  (PLATO  and 
CASCADE-SEA).  However,  it  should  be  noted  that  most  of  these  claims  appear  to  remain  assumptions, 
since  few  data  are  available  that  demonstrate  the  actual  benefits  of  these  tools. 


C.  Type  of  Process  Supported  and  Underlying  Theory 

Many  tools  that  were  analyzed  make  extensive  use  of  a prototyping  approach,  which  refers  to  a 
pragmatic  paradigm.  This  is,  with  the  exception  of  GAIDA,  Mercator,  IDXelerator  and  GTE:  these  tools 
seem  to  be  based  on  a paradigm  that  follows  a more  linear  completion  of  the  instructional  design  process. 

When  looking  closer  at  the  underlying  elements  of  the  systematic  approach  to  development  of 
education  and  training  it  appears  that  two  tools  (QIPP  and  CASCADE-SEA)  intend  to  support  the  designer 
during  the  entire  process  (from  analysis  through  evaluation).  All  other  tools  support  specific  elements  of 
the  process,  of  which  design  and  development  get  the  most  attention.  When  reviewing  the  tools  with 
respect  to  the  underlying  teaching/leaming  theory,  it  appears  that  most  tools  are  based  (to  various  degrees) 
on  a cognitivistic  theory.  Two  tools  seem  to  be  based  on  a more  behavioristic  theory  (GAIDA  and 
IDXelerator)  and  one  starts  from  constructivism  (MediaPlant). 


D.  Task  Support 

All  tools  contain  job  aids  to  support  users  in  their  development  activities.  The  metaphor  of  a 
toolbox  and  a do-it-yourself  kit  fits  most  tools.  None  of  the  systems  that  were  analyzed  have  the  ability  to 
automate  the  entire  instructional  design  process.  In  all  cases,  considerable  human  skills  are  needed  to  make 
effective  instructional  products  and  courses.  It  is  noteworthy  that  none  of  the  tools  seem  to  include  explicit 
learning  facilities  for  designers  who  express  a need  for  learning  a specific  design  task.  For  novice 
designers,  with  the  possible  exception  of  GAIDA,  the  tools  seem  to  count  on  an  informal  learning  process 
of  leaming-by-doing  or  some  form  of  external  assistance. 


E.  Intended  User  Group 

Generally  speaking,  it  appears  the  designers  of  all  tools  started  creating  the  tools  with  a specific 
organization  in  mind.  In  an  overall  view,  the  tools  are  intended  for  one  or  two  of  the  following  user  groups: 
professional  designers,  subject  matter  experts,  teachers  and/or  learners.  This  means  that  the  support  tools 
need  to  contain  (parts  of)  team  members’  expertise  that  would  have  been  needed  in  times  when  the  tool  was 
not  available. 


3.  Trends  in  Computer  Supported  Curriculum  Development 
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In  looking  ahead,  there  are  several  trends  that  will  impact  the  form  and  substance  of  future 
computer-based  tools  for  developing  education  and  training. 


Supporting  a Constructivist  Perspective  on  Learning 

The  increased  influence  of  the  constructivist  perspective  on  learning  is  impacting  the  design  of 
computer-based  support  tools.  From  this  perspective,  learning  requires  active  construction  rather  than 
acquisition  of  knowledge  by  the  learner.  As  a consequence,  the  teacher  will  increasingly  become  a designer 
of  learning  environments  that  support  the  construction  of  knowledge  of  the  learners.  Also,  teachers  or 
trainers  increasingly  fulfil  design  roles  in  the  context  of  innovative  projects,  in  which  they  participate,  often 
emphasizing  their  own  professional  development. 


Increasing  Array  of  Tools 

The  first  tools  were  almost  all  created  to  support  only  one  or  a few  tasks  related  to  the  curriculum 
design  process.  Although  there  are  now  some  tools  that  support  many  different  tasks,  none  are  completely 
adequate  for  all  tasks  on  different  types  of  projects.  What  we  now  see  is  expansion  in  several  directions: 
there  is  an  increase  in  the  number  of  tools  that  attempt  to  integrate  multiple  tasks,  however,  at  the  same 
time  the  number  of  single  purpose  tools  for  highly  specialized  situations  is  also  increasing.  Continuing 
advances  in  computers,  digital  processing,  and  communication  technology  will  all  add  to  the  demand  for  a 
complementary  set  of  development  tools  and  support  new  features  related  to  future  design  efforts. 


Supporting  Teamwork 

Team  efforts  are  increasingly  critical  to  large  scale,  complex  projects,  especially  those  that  will 
result  in  technology-based  instruction  such  as  multimedia  or  web-based  course.  As  a consequence, 
computer-based  support  tools  may  be  extended  with  communication  tools  that  facilitate  collaboration.  In 
addition,  anticipation  of  how  the  curriculum  intervention  will  be  implemented  is  of  growing  significance 
during  the  design  process. 


Supporting  Networks  of  Designers 

As  individual  designers  gain  knowledge  and  skill  in  using  the  tools  they  can  more  readily  share 
this  knowledge  with  other  members  of  the  design  community  to  prevent  these  insights  and  skills  from 
being  lost  to  others  or  not  be  otherwise  leveraged  in  the  organization.  Based  on  today’s  database  and 
networking  technologies,  effective  computer-based  infrastructures  may  be  developed  which  makes 
knowledge  sharing  and  knowledge  management  more  possible. 


For  all  of  these  reasons,  we  believe  the  future  of  computer-based  support  tools  is  very  bright.  The 
emergence  and  expansion  of  tool  creation  and  use  that  we  have  witnessed  over  the  last  ten  years  will  pale  in 
comparison  to  what  will  happen  during  the  next  decade.  We  have  no  doubt  that  future  tools  will  be  as 
different  from  current  ones  as  current  desktop  computers  are  from  their  predecessors  of  ten  years  ago. 
Continuing  advances  in  computers,  digital  processing,  and  communication  technology  will  both  add  to  the 
demand  for  a complementary  set  of  development  tools  and  support  new  features  we  can  only  dream  about 
today. 


206 


720 


References 


O 

ERIC 


Carr,  C.  (1992).  Performance  support  systems:  The  next  step?  Performance  & Instruction,  31(2), 

23-26. 

Collis,  B.,  Boer,  W.  de  (1999).  The  TeleTOP  decision  support  tool  (DST).  In  J.  Van  den  Akker, 
R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design  approaches  and  tools  in  education  and 
training  (pp.  235-248).  Dordrecht:  Kluwer. 

Geber,  B.  (1991).  Help!  The  rise  of  performance  support  systems.  Training,  28(12),  23-29. 

Gery,  G.J.  (1991).  Electronic  performance  support  systems:  How  and  why  to  remake  the 
workplace  through  strategic  application  of  technology.  Boston,  MA:  Weingarten. 

Gettman,  D.,  McNelly,  T.,  & Muraida,  D.  (1999).  The  guided  approach  to  instructional  design 
advising  (GAIDA):  A case-based  approach  to  developing  instructional  design  expertise.  In  J.  Van  den 
Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design  approaches  and  tools  in  education 
and  training  (pp.  175-182).  Dordrecht:  Kluwer. 

Jong,  T.  De,  Limbach,  R.,  Gellevij,  M.,  Kuyper,  M.,  Pieters,  J.,  & Joolingen,  W.  van  (1999). 
Cognitive  tools  to  support  the  instructional  design  of  simulation-based  discovery  learning  environments: 
The  SimQuest  authoring  system.  In  J.  Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj. 
Plomp.  Design  approaches  and  tools  in  education  and  training  (pp.  215-224).  Dordrecht:  Kluwer. 

Jury,  T.  & Reeves,  T.  (1999)  An  EPSS  for  instructional  design:  NCR’s  quality  information 
products  process.  In  J.  Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design 
approaches  and  tools  in  education  and  training  (pp.  183-194).  Dordrecht:  Kluwer. 

Marcke,  K.  Van  (1998).  GTE:  An  epistomological  approach  to  instructional  modeling. 
Instructional  Science,  26(3/4),  147-191. 

McGraw,  K.L.  ( 1 995).  Selecting  appropriate  functionality  and  technologies  for  EPSS.  Innovations 
in  Education  and  International  Training,  32(1),  13-22. 

McKenney,  S.  (1999).  CASCADE-SEA:  Computer  assisted  curriculum  analysis,  design  and 
evaluation  for  science  education  in  Africa.  In  J.  Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & 
Tj.  Plomp.  Design  approaches  and  tools  in  education  and  training  (pp.  225-234).  Dordrecht:  Kluwer. 

Merrill,  D.  Thompson,  B.  (1999).  The  IDXelerator:  Learning-centered  instructional  design.  In  J. 
Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design  approaches  and  tools  in 
education  and  training  (pp.  265-278).  Dordrecht:  Kluwer. 

Nieveen,  N.,  & Gustafson,  K.  (1999).  Characteristics  of  computer-based  tools  for  education  and 
training  development:  An  introduction.  In  J.  Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj. 
Plomp.  Design  approaches  and  tools  in  education  and  training  (pp.  155-174).  Dordrecht:  Kluwer. 

Paquette,  G.,  Aubin,  C.,  & Crevier,  F.  (1994).  An  intelligent  support  system  for  course  design. 
Educational  Technology,  34(9),  50-57. 

Preese,  F.,  & Foshay,  Q.  (1999).  The  PLATO  courseware  development  environment.  In  J.  Van 
den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design  approaches  and  tools  in 
education  and  training  (pp.  195-204).  Dordrecht:  Kluwer. 

Raybould,  B.  (1995).  Performance  support  engineering:  An  emerging  development  methodology 
for  enabling  organizational  learning.  Performance  Improvement  Quarterly,  8(1),  7-22. 

Rossett,  A.,  & Gautier-Downes,  J.  (1991).  A handbook  of  job  aids.  San  Diego:  Pfeiffer. 

Spector,  M.  ( 1 999).  Intelligent  support  for  instructional  development:  Approaches  and  limits.  In  J. 
Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design  approaches  and  tools  in 
education  and  training  (pp.  279-290).  Dordrecht:  Kluwer. 

Valcke,  M.,  Kirschner,  P,  Bos,  E.  (1999).  Enabling  technologies  to  design,  produce  and  exploit 
flexible,  electronic  learning  materials.  In  J.  Van  den  Akker,  R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj. 
Plomp.  Design  approaches  and  tools  in  education  and  training  (pp.  249-264).  Dordrecht:  Kluwer. 


207 


Wright,  R.,  Harper,  B.,  & Hedberg,  J.  (1999).  Visual  support  for  authoring.  In  J.  Van  den  Akker, 
R.M.  Branch,  K.  Gustafson,  N.  Nieveen,  & Tj.  Plomp.  Design  approaches  and  tools  in  education  and 
training  (pp.  205-214).  Dordrecht:  Kluwer. 


208 


722 


Influences  of  Concept  Mapping  and  Learning  Styles  on  Learning 


Ellen  Taricani 
Penn  State  University 


The  effect  of  concept  mapping  with  different  levels  of  knowledge  to  assist  those  with  different  learning 
styles  will  be  discussed.  This  paper  will  discuss  the  methods  used  to  compare  these  influences  and  the 
differences  in  particular  learning  processes. 

Concept  mapping 

Concept  mapping  is  a technique  for  visually  representing  the  structure  of  information,  concepts 
and  the  relationships  between  them.  They  can  be  compared  to  road  maps  that  help  us  find  direction  in  the 
midst  of  numerous  signs,  roads,  and  other  distractions.  Substantial  changes  in  the  complexity  of  the 
knowledge  structures  take  students  beyond  rote  learning  toward  meaningful  learning.  Students  using 
concept  maps  increase  their  domain  knowledge  (Ruiz-Primo  & Shavelson,  1996)  toward  applications  and 
assessments. 

Concept  maps  are  useful  tools  that  help  students  learn  about  how  they  structure  knowledge  while 
supporting  the  process  of  knowledge  construction  or  metaknowledge.  In  this  way,  concept  maps  help 
students  learn  how  to  learn  commonly  referred  to  as  metaleaming.  A concept  map  requires  the  learner  to 
function  at  all  four  levels  of  Merrill’s  (1983)  content  dimensions:  fact,  concept,  procedure,  and  principle. 
Each  of  these  dimensions  identifies  three  performance  levels:  remember,  use,  and  find.  "Remember" 
causes  the  students  to  search  their  memory  in  order  to  reproduce  or  recognize  a previously  stored  item. 
"Use"  will  require  the  student  to  apply  some  abstraction  to  a specific  case.  Use  means  to  use  a general  rule 
to  process  specific  information.  Finally,  "find"  is  the  performance  level  that  the  student  begins  to  derive  or 
invent  a new  abstraction. 

The  concept  map  becomes  an  external  representation  presenting  internal  processes  of  information 
in  structured  graphs  (Jonassen,  Beissner  & Yacci,  1 993).  It  becomes  a representation  of  the  application  of 
the  dimensions  of  conceptual  processes  at  different  content  levels.  The  nodes  aid  links  represent 
relationships  between  the  concepts  and  demonstrate  the  depth  of  processing  knowledge.  Conceptual 
understanding  can  be  described  as  the  richness  of  interconnections  and  relationships  made  between 
concepts  and  the  structure  that  organizes  those  concepts  (Novak  & Gowin,  1984). 

Concept  maps  have  been  used  successfully  to  promote  learning.  How  do  students  internalize  and 
organize  readings,  assignments  and  notes  to  prepare  a concept  map  as  a study  tool  for  a test  that  evaluates 
different  levels  of  learning?  How  do  they  process  new  facts,  concepts,  etc.,  to  complete  assignments  and 
take  tests?  Can  a concept  map  be  a significant  factor  in  facilitating  different  levels  of  learning?  Will 
different  learning  styles  affect  the  strategies  used  in  creating  a concept  map  and  the  follo\wthrough  when 
given  a test  on  the  subjects? 

Learning  Styles  / Cognitive  Styles 

Attempts  to  describe  patterns  of  information  processing  have  led  to  the  development  of  several 
theories  in  the  field  of  educational  psychology  (Noring,  1993),  but  most  commonly  these  approaches  have 
been  divided  into  two  broad  classes  of  learning  styles  (also  called  cognitive  styles,  see  Kearsley,  1994), 
known  as  field  dependent  and  field  independent.  Field  dependent  learners,  the  most  prevalent,  need  to  see 
each  segment  of  instruction  in  relation  to  the  preceding  instruction  and  the  overall  aims  of  the  course.  Field 
independent  learners,  on  the  other  hand,  are  apt  to  find  their  own  structure  in  which  to  place  the  instruction 
and  other,  extraneous  applications  for  the  instruction.  Whereas  field  dependent  students  learn  best  the 
material  presented  within  its  social  context,  field  independent  learners  prefer  a more  clinical,  analytical 
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presentation  of  material,  and  learn  social  material  more  as  an  intentional  task  than  a natural 
response.  In  place  of  the  external  goals  and  reinforcements  desired  by  the  field  dependent  learner,  the  field 
independent  student  values  knowledge  for  its’  own  sake,  and  often  has  a personal  set  of  goals  and  rewards 
to  strive  for. 

Differences  in  cognitive  styles  do  not  indicate  differences  in  learning  ability  or  memory  (Witkin, 
Moore,  Goodenough,  and  Cox,  1977).  Cognitive  styles  indicate  the  preferences  that  an  individual  has  for 
perceiving  and  processing  information,  not  the  ability  to  learn  the  material.  Thus,  students  with  equal 
learning  abilities  but  different  cognitive  styles  may  experience  different  levels  of  success  in  the  same 
environment. 

Meaningful  Learning 

The  idea  behind  concept  maps  was  derived  from  Ausubel’s  theory  of  meaningful  versus  rote 
learning.  Meaningful  learning  occurs  when  students  intentionally  attempt  to  integrate  new  knowledge  with 
existing  knowledge.  A learner  who  attempts  to  integrate  knowledge  will  most  likely  have  a more  extensive 
network  of  knowledge  and  therefore  more  retrieval  paths.  His  subsumption  theory  involves  reorganization 
of  existing  cognitive  structures  not  development  of  new  structures.  A primary  process  in  learning  is 
subsumption  in  which  new  material  is  related  to  relevant  ideas  in  the  existing  cognitive  structure  on  a 
substantive,  non-verbatim  basis.  Cognitive  structures  represent  the  residue  of  all  learning  experiences; 
forgetting  occurs  because  certain  details  get  integrated  and  lose  their  individual  identity. 

Rote  learning  occurs  because  a student  simply  memorizes  information  with  no  attempt  or 
motivation  to  relate  that  information  to  prior  knowledge.  Therefore,  the  rote  learner  will  have  a less 
extensive  network  than  the  meaningful  learner  and  less  retrieval  paths  between  knowledge  concepts. 
Concept  maps  are  one  way  to  foster  and  measure  meaningful  learning  in  the  classroom  as  instructional, 
student  learning,  and  assessment  techniques. 

Instructionally,  concept  maps  foster  meaningful  learning  by  teaching  the  connections  among 
course  concepts.  As  a student  learning  tool,  concepts  maps  promote  meaningful  learning  by  encouraging 
the  students  to  generate  their  own  connections  between  concepts.  In  terms  of  assessment,  concept  maps 
evaluate  if  and  how  meaningful  learning  is  occurring. 

According  to  Ausubel's  (1963)  Meaningful  Learning  Theory,  we  build  meaning  every  time  we 
establish  substantive  rather  than  arbitrary  relationships  between  the  study  material  and  existing  knowledge. 
When  students  encounter  new  material  they  approach  it  from  a series  of  concepts  and  representations 
acquired  from  previous  experiences.  These  experiences  are  used  as  instruments  of  interpretation  that 
partially  determine  what  information  the  students  will  absorb,  how  they  will  organize  the  information,  what 
types  of  relationships  they  will  establish  among  the  pieces  of  information,  what  problem-solving  techniques 
they  will  use,  and  so  on. 

This  explains  why  the  concepts  do  not  represent  the  same  for  the  teacher  as  for  the  student — the 
concepts  have  neither  the  same  relevancy  nor  the  same  explanatory  power.  Ausubel  argues  that  when 
discipline  is  taught,  it  fundamentally  transmits  this  conceptual  structure  to  the  students. 

The  appropriation  of  complex  structures  of  knowledge  implies  an  understanding  of  them,  and  that 
understanding  cannot  be  reached  only  by  routine  procedures.  The  acquisition  and  retention  of  a body  of 
knowledge  implies  the  assimilation  of  a body  of  conceptual  meanings — the  product  of  meaningful  learning. 

In  Ausubel's  words,  concepts  are  acquired  by  progressive  differentiation — that  is,  those  concepts 
that  are  ordered  in  a hierarchy  that  progresses  from  the  most  general  to  the  most  specific  idea.  New 
information  is  assimilated  into  existing  conceptual  hierarchies  in  the  cognitive  structure.  These 
modifications  are  not  merely  juxtapositions  of  concepts,  because  the  final  meaning  of  a structure  is  not 
equivalent  to  the  sum  of  the  parts — it  forms  a new  structure. 
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In  the  psychological  structure,  a related  process  of  integrative  reconciliation  occurs  that  allows 
knowledge  to  relate  to  the  discipline  and  modify  preconceptions  or  misconceptions,  thus  reducing 
fragmentation  and  making  possible  a reflective  and  critical  attitude. 

The  existing  structure  of  knowledge  influences  the  capacity  to  interpret  reality  and  to  take  part  in 
it.  It  creates  a framework  that  will  open  or  shut  depending  on  the  individual's  capacity  to  understand.  This 
capacity  to  approach  and  solve  problems  depends  on  the  density  of  meanings  in  the  existing  structure  of 
knowledge.  There  will  be  dominant  areas  in  which  the  effect  from  an  experience  is  quite  broad  and  for 
which  the  structure  of  meanings  is  exceptionally  powerful,  and  there  will  be  others  in  which  precisely  the 
opposite  happens. 

From  this  perspective,  attending  to  the  potential  of  the  study  material  to  develop  thought  skills  implies 
two  things.  First,  the  knowledge  must  be  organized  with  the  discipline  and  its  methodology  following  a 
hierarchical  relation  scheme  that  is  part  of  the  most  general  and  most  inclusive  concepts  of  the  material  and 
advances  toward  the  most  particular.  This  descending  cyclical  sequencing  allows  us  to  put  relief  in 
different  relationships  that  maintain  the  concepts  among  themselves.  Effective  linkage  results  in 
subsumption;  that  is,  new  knowledge  is  meaningfully  joined  with  other  knowledge  in  the  student's 
cognitive  structure.  Second,  it  must  facilitate  the  assimilation  of  concepts  (the  progressive  differentiation 
and  the  integrative  reconciliation)  through: 

• the  initial  presentation  of  ^neral  ideas  that  provide  one  conceptual  framework  for  subsequent 
knowledge 

• using  specific  examples  in  real  contexts  that  illustrate  the  concepts  and  their  relationships  in  such  a 
way  that  they  acquire  meaning  and  feeling 

• the  combination  and  the  sequence  of  positive  and  negative  examples  that  facilitate  the  conceptual 
differentiation 

• representation  of  the  knowledge  in  graphical  systems,  such  as  the  concept  maps,  that  help  better 
understand  the  relationships  among  the  ideas  and  the  procedures 

Progressive  dfferentiation  is  presenting  the  most  general  and  inclusive  ideas  first,  followed  by 
increasing  detail.  Integrative  reconciliation  is  pointing  out  similarities  and  differences  between  old  and  new 
learning.  Cognitive  strategies  are  skills  that  may  aid  the  learner  in  an  internal  process  of  attending, 
selective  perceiving,  coding  for  long  term  storage,  retrieval  and  problem  solving. 

Concept  mapping  is  a technique  for  visually  representing  the  structure  of  information — concepts  and 
the  relationships  between  them — and  can  therefore  aid  the  student  in  the  process  of  meaningful  learning. 
Concept  mapping  is  also  designed  to  encourage  the  mapper  to  make  his  or  her  own  connections  among 
knowledge  concepts.  The  more  meaningful  connections  a person  can  show  in  the  map,  the  better  s/he  will 
understand  the  material.  The  process  of  mapping  a map  and  the  final  product  are  dependent  on  prior 
knowledge,  context,  and  constructed  understanding. 

In  constructivism,  active  and  meaningful  learning  is  encouraged  as  an  educational  philosophy. 
Learning  is  a structuring  process  where  knowledge  is  derived  from  experiences.  Ausubel  argued  that 
learning  new  materials  depends  greatly  on  the  existing  cognitive  structure  or  what  the  person  already 
knows.  New  information  will  be  more  easily  learned  if  it  is  explained  and  also  related  to  relevant  ideas  in 
the  student's  cognitive  structure.  Meaningftil  learning  occurs  when  new  information  is  linked  to  prior 
information  in  the  learner's  own  cognitive  structure.  New  information  is  more  meaningful  if  it  is  related  to 
existing  knowledge. 

The  basis  for  this  active  learning  role  can  best  be  described  in  Ausubel’s  (1963)  theory  of  cognitive 
learning.  AusubeFs  principles  of  non-arbitrary  assimilation  of  knowledge  through  concept  differentiation 
and  concept  integration  have  become  the  fundamental  principles  for  meaningful  learning  theory 
(Wandersee,  1990).  Meaningful  learning  theory  is  based  on  these  principles: 

1)  knowledge  is  stored  hierarchically  in  idiosyncratic  cognitive  structures; 
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2)  prior  knowledge  influences  new  learning,  although  misconceptions  are  acquired  early  and  are 
resistant  to 

change; 

3)  humans  construct  new  concepts  and  propositions  through  meaningful  learning; 

4)  meaningful  learning  is  at  one  end  of  a learning  continuum  with  rote  learning  at  the  other 


Component  Display  Theory 

Component  Display  Theory  (CDT)  classifies  learning  along  two  dimensions:  content  (facts, 
concepts,  procedures,  and  principles)  and  performance  (remembering,  using,  and  generalities).  The  theory 
specifies  four  primary1  presentation  forms:  rules  (expository  presentation  of  a generality),  examples 
(expository’  presentation  of  instances),  recall  (inquisitory  generality)  and  practice  (inquisitory’ instance). 
Secondary  presentation  forms  include:  prerequisites,  objectives,  helps,  mnemonics,  and  feedback.  Merrill 
(1983)  explains  the  assumptions  about  cognition  that  underly  CDT.  While  acknowledging  a number  of 
different  types  of  memory >,  Merrill  claims  that  associative  and  algorithmic  memory  structures  are  directly 
related  to  the  performance  components  of  Remember  and  Use/Find  respectively.  Associative  memory  is  a 
hierarchial  network  structure;  algorithmic  memory  consists  of  schema  or  rules.  The  distinction  between 
Use  and  Find  performances  in  algorithmic  memory  is  the  use  of  existing  schema  to  process  input  versus 
creating  a new  schema  through  reorganization  of  existing  rules. 

Component  Display  Theory  is  concerned  with  teaching  individual  concepts  or  principles, 
classifies  objectives  on  two  dimensions,  and  formats  instruction  to  provide  learner  control.  Component 
Display  Theory  is  composed  of  three  parts  (Merrill,  1983): 

• A performance/content  matrix  that  includes  the  desired  level  of  student 
performance  (Remember  Instance,  Remember  Generality,  Use,  and  Find)  and 
the  type  of  content  (Fact,  Concept,  Procedure,  and  Principle) 

• Four  primary  presentation  forms:  Expository  (Rule,  Example)  and  Inquisitory 
(Recall,  Practice) 

• A set  of  prescriptions  relating  the  level  of  performance  and  type  of  content  to  the 
presentation  forms 

Instruction  will  be  more  effective  if  all  three  primary  performance  forms — remember,  use,  and  find — 
are  present  for  the  different  types  of  content  or  information  types.  Primary  forms  can  be  presented  by 
either  an  explanatory  or  inquisitory  learning  strategy.  The  sequence  of  primary  forms  is  not  critical 
provided  they  are  all  present.  A concept  map  can  be  used  to  provide  examples  of  the  concepts  as  the 
learner  elaborates  on  that  particular  concept. 

Principles  of  CDT 

1 . Instruction  will  be  more  effective  if  all  three  primary  performance  forms  (remember,  use,  generality)  are 
present. 

2.  Primary  forms  can  be  presented  by  either  an  explanatory  or  inquisitory  learning  strategy 

3.  The  sequence  of  primary  forms  is  not  critical  provided  they  are  all  present. 

4.  Students  should  be  given  control  over  the  number  of  instances  or  practice  items  they  receive. 

Using  concept  mapping  provides  a form  of  both  remember  and  use.  Students  are  asked  to  apply 
what  they  are  learning  as  they  draw  the  map.  The  generality  is  applied  in  the  testing  process.  They  are 
given  control  over  their  instances  as  they  create  their  individual  maps.  Knowledge  objects  can  be  linked 
via  component  relationships:  an  entity  can  be  a part  of  another  entity,  an  activity  can  be  a step  of  another 
activity,  or  a process  can  be  an  event  of  another  process.  The  learner  acquires  knowledge  of  discriminating 
properties  and  is  able  to  sort  instances  with  respect  to  these  discriminating  properties.  CDT  identifies 
strategy  prescriptions  for  different  kinds  of  learning  outcomes.  Components  of  instruction  provides  a way 
to  more  precisely  analyze  subject  matter  content  and  more  precisely  design  instructional  strategies  to 
present  this  material.  A significant  aspect  of  the  CDT  framework  is  learner  control.  This  idea  enables 
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learners  to  select  their  own  instructional  strategies  in  terms  of  content  and  presentation  components.  In  this 
sense,  instruction  designed  according  to  CDT  provides  a high  degree  of  individualization  since  students  can 
adapt  learning  to  meet  their  own  preferences  and  styles. 

Two  types  of  knowledge  are  part  of  the  levels  of  learning  that  Merrill  describes.  Concepts  are 
categories  of  experience  bounded  by  a definition  and  given  in  a name.  They  are  groups  or  classes  of  things 
that  have  something  in  common.  Concepts  are  arbitrary  groupings  that  are  invented  by  people.  This  is 
displayed  by  key  terms,  items,  processes,  and  categorizing  items.  An  important  consideration  for  learning 
concept  classification  is  prior  knowledge.  Robert  Gagne  (1985)  has  shown  that  the  skill  of  applying  a 
concept  always  has  some  "prerequisite"  skills,  skills  that  must  be  mastered  before  it  is  possible  to  learn  any 
given  classification  skill.  A principle  is  a relationship  among  factors.  It  is  composed  of  two  or  more 
concepts  having  an  ordered  relationship  to  each  other  (Gagne,  1971).  Students  display  this  knowledge  by 
applying  the  principle.  Some  examples  are:  control,  predict,  infer  cause,  explain,  troubleshoot,  vary  the 
task  based  on  new  conditions.  There  are  two  phases  to  learning  a principle  at  the  application  level.  The 
learner  needs  to  comprehend  the  principle.  This  is  referred  to  as  the  acquisition  phase.  After  it  is  acquired, 
the  learner  needs  to  learn  to  generalize  it  to  new  situations,  which  is  called  the  application  phase. 

The  instructional  sequence  to  be  used  is  based  upon  performance  levels  designed  by  Merrill.  His 
model  suggests  that  the  instructional  sequence  should  include  application  and  examples  of  each  order  of 
learning:  remember,  use  and  find.  This  involves  establishing  an  instructional  strategy  for  each  phase.  I 
will  add  another  tool,  feedback,  to  the  sequence  to  improve  learning. 


Order 

Instructional  sequence 

1 . remember 

Provide  the  learner  with  examples  that  lead  to  an 
expectation  of  successful  accomplishment  of  the 
skill 

2.  use 

Give  learners  the  responsibility  to  practice  and  use 
the  new  skill  in  a supportive  learning  environment 

3.  find 

New  examples  will  be  added  to  the  concept  maps 

4.  feedback 

Provide  feedback  through  expert  maps  to  learners  on 
how  to  improve  their  concept  maps  based  on 
instruction  on  concepts  and  principles 

This  Study 

In  this  study  students  were  asked  to  create  concept  mapping  and  some  were  given  feedback  and 
instruction  to  assist  in  their  knowledge  integration.  Effects  of  different  treatments  were  established  with 
the  students  to  compare  the  differences  with  feedback  and  extra  training  on  knowledge  application.  After 
reviewing  the  literature  and  issues  raised,  the  researcher  has  concluded  that  the  use  of  concept  mapping  can 
assist  students  in  separating  information  and  organization  of  it  for  recall.  Concept  mapping  does  assist  in 
learning  and  developing  skills  for  meaningful  learning.  However,  does  learning  concept  mapping  with 
instruction  on  concepts  and  principles  with  feedback  result  in  more  learning  for  field  dependent  learners? 
Does  a field  independent  learner  do  better  without  feedback  on  concept  mapping  and  instruction  on 
concepts  and  principles  since  they  tend  to  favor  less  structure?  The  purpose  of  this  study  is  to  determine  if 
more  learning  takes  place  when  using  concept  mapping  with  instruction  on  concepts  and  principles,  with  or 
without  feedback.  In  addition,  the  purpose  of  the  study  is  to  determine  if  field  dependent  learners  tend  to 
do  better  with  structure,  framed  by  instruction  on  concepts  and  principles,  than  field  independent  learners. 

Summary 

Concept  mapping  is  tool  that  can  be  used  for  facilitating  learning  and  assessing  meaningful 
learning.  It  helps  in  gaining  better  and  more  comprehensive  understanding  of  learning  information.  This 
study  will  attempt  to  determine  if  using  concept  maps  will  have  a significant  effect  on  both  concept-type 
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and  principle-type  learning.  Concept  maps  serve  to  clarify  links  between  new  and  old  knowledge  giving  an 
opportunity  for  the  learner  to  externalize  the  links  through  test  taking.  In  conclusion,  this  study  will  look  at 
what  can  be  done  to  help  students  create  more  effective  concept  maps  using  treatments  such  as  instruction 
on  concept  mapping  and  instruction  on  concepts  and  principles,  with  or  without  feedback,  to  assisi  inem  in 
better  learning  techniques. 
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Abstract 


Virtual  resources  centers  have  been  considered  a pedagogical  tool  since  the  increasing 
development  of  electronic  means  allowed  the  storage  of  luge  amounts  of  information  and 
their  easy  retrieval.  Bearing  in  mind  the  need  for  enhancing  the  appearance  of  those  centers,  a 
discipline  of  "Management  of  Resources  Centers"  was  included  in  a specialized  diploma 
course  run  at  the  University  of  Minho  (Portugal)  for  irnservice  primary  teachers.  As  a final 
work  for  that  course,  one  of  us  (A.  Silva)  developed  a virtual  resources  center  about  Ponte  de 
Lima,  which  can  be  easily  accessed  through  the  Web  site  meanwhile  created  - 
(http://www.pontedelima.com).  The  center  has  not  merely  educational  purposes;  it  also  has  in 
mind  simple  local  information  useful  for  tourism  or  even  to  get  native  people,  living  outside, 
in  touch  with  their  motherland.  We  describe  the  way  the  site  was  conceived  and  how  its  main 
areas  developed;  additionally,  wc  present  some  statements  of  users,  looking  for  understanding 
of  how  they  benefited  from  this  tool  now  at  their  service. 

Introduction 


Since  the  mid-sixties  Portugal  has  adopted  deliberately  an  educational  policy  where 
instructional  technology  played  a significant  role.  Although  at  this  time  the  country  was  ruled  by  the 
retrograde  dictatorship  of  the  Prime  Minister  Oliveira  Salazar9,  some  signs  of  openness  regarding  the 
education  could  be  noticed,  specially  concerning  the  principle  of  extending  the  education  opportunities. 

Portugal  is  a mountainous  country,  particularly  in  the  Northern  part.  There  are  hundreds  of 
very  small  villages  that  stand  sparse  in  the  valleys  and  even  in  the  mountains.  At  this  time,  the  road 
network  was  a very  bad  one.  One  of  the  wiser  decisions  of  Oliveira  Salazar  was  to  build  small 
elementary  schools  (grades  1-4)  throughout  the  country,  even  in  the  smallest  places.  However,  while 
the  first  four  years  of  schooling  were  assured,  the  subsequent  years  were  not,  be  cause  of  the  distance  to 
a major  center  and  the  extreme  poverty  of  the  people  living  in  distant  villages,  who  could  not  pay 
additional  housing  expenses.  This  was  the  main  reason  why  most  students  did  not  take  further  courses 
at  the  secondary  schools. 

In  1965  the  Government  decided  to  set  up  the  IMAVE  (fnstituto  de  Meios  Audio-Visuais  de 
Ensino  - Institute  of  Audio-Visual  Means  for  Teaching)  and  also  the  Telescola  (TV  school).  The 
Telescola  started  running  courses  by  television  to  provide  students  in  rural  schools  two  more  years  (5- 
6)  of  schooling,  that  is  to  say,  extending  from  four  to  six  the  number  of  school  years.  The  design  of 
Telescola , as  well  as  its  implementation,  was  a very  good  one.  Besides  the  short  lessons  by  television 
in  each  school  an  elementary  teacher  acted  as  a monitor,  adding  some  direct  teaching  to  the  televised 
one.  As  is  said  in  an  OECD  evaluative  report,  showing  high  regard  for  the  Portuguese  innovation, 

[i]t  was  altogether  unexpected  that  out  of  Portugal  should  have  emerged  the 
only  well  established  example  in  Europe  of  an  integrated  learning  system,  in 
which  television  played  a central  instructional  role,  covering  the  full 
curriculum  at  ‘first-cycle  secondary’  level,  and  dealing  with  tens  of 
thousands  of  children  (OECD,  1977,  p.  2). 


Oliveira  Salazar  was  the  Prime  Minister  of  Portugal  between  1928  and  1968,  and  ruled  the  country  as  a dictator. 
His  successor,  Marcelo  Caetano  (1 968-1974),  though  more  moderate,  followed  the  main  lines  of  Salazar’s  policy 
and  was  overthrown  by  the  25,h  April  1974  Revolution  (the  Revolution  of  Carnations). 
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The  Telescola  certainly  was  the  most  evident  exploit  in  instructional  technology  in  Portugal. 
At  its  peak  tens  of  thousand  students  followed  yearly  its  courses.  Still  today  the  Telescola  continues  to 
serve  some  villages  where  it  is  difficult  to  implement  post-elementary  schools.  Of  course  Portugal  is 
no!  today  as  it  was  in  the  1 960’s.  On  the  one  hand,  most  small  villages  lost  young  population  and  today 
there  are  no  children  to  go  to  school:  some  hundreds  of  primary  schools  closed.  On  the  other  hand,  road 
conditions  improved  a great  deal,  and  the  isolation  was  broken  for  a majority  of  places. 

However,  it  is  still  difficult  to  teachers  living  in  small  villages  to  cope  with  the  lack  of 
information  and  pedagogical  materials  they  need  in  their  everyday  life.  Teachers  recurrently  complain 
about  their  situation;  they  want  and  they  deserve  to  have  much  more  support.  Several  strategies  have 
been  thought  of  to  minimize  the  poor  conditions  teachers  face  in  the  rural  world.  Once  more 
technology  seems  to  be  the  best  way  to  reach  such  a goal:  the  Internet  may  help  those  teachers,  due  to 
the  establishment  of  virtual  resources  centers. 

What  the  Literature  Says 

Resource  centers  are  not  a novelty:  even  in  the  older  schools,  some  educational  materials  were 
kept,  having  in  mind  their  re-utilization  by  teachers  and  students  to  improve  the  learning  process.  In  the 
beginning,  however,  school  libraries  stood  as  the  main  documentation  centers.  As  the  audio-visual 
materials  developed,  school  libraries  became  true  school  media  centers,  keeping  together  books  and 
other  script  materials  with  films,  slides,  audio  and  videocassettes. 

In  1972,  the  American  Association  of  School  Librarians  (AASL)  acknowledged  the  shifting 
times,  deciding  to  change  the  name  of  the  AASLjoumal,Sc/ioo/  Libraries , to  School  Media  Quarterly . 
In  a rejoicing  article,  “A  title  for  the  times”,  Srygley  reports  us  that  ‘tf]or  more  than  twenty  years 
AASL  has  given  strong  leadership  in  defining  the  school  library  as  a media  center,  learning  resources 
center,  or  instructional  materials  center”  (1972,  p.  16).  Therefore,  we  can  assume  that  almost  thirty 
years  ago  the  label  “resource  center”  came  to  the  everyday  life  of  schools. 

The  development  of  computer  technology  determined  that  even  the  professionals 
who  were  reluc  tant  to  admit  a “library”  keeping  materials  other  than  books  and  journals,  should  change 
their  minds.  In  1983,  Koskiala  summed  up  the  new  librarian  point  of  view:  “Nonprint  is  beginning  to 
be  given  serious  consideration  in  some  accredited  programs”  (1983,  p.  309). 

The  role  of  media  specialists  is  going  to  change,  and  fast:  in  an  examination  of  the 
impact  of  the  new  information  age  on  education,  Considine  told  us  the  sincere  statement  of  one 
administrator,  saying  that  “media  specialists  were  an  endangered  species,  threatened  with  extinction 
because  they  were  neither  visible  nor  viable”  (1985,  p.  182).  He  added:  “For  media  specialists  to 
survive  such  a threat,  they  must  change  not  only  their  role  but  the  perception  many  teachers  and 
administrators  have  of  that  role”  (1985,  p.  182).  Later,  Schiffman  (1987)  wrote  about  the  “window  of 
opportunity”  open  to  the  school  library  media  centers,  entering  in  the  online  environment,  to  change 
public  education. 

Other  authors  (e.g.,  Craver,  1986;  Dede,  1985;  Ely,  Blair,  Lichvar,  Tyksinski,  & Martinez, 

1 996)  drew  readers’  attention  to  the  effects  of  the  computer  explosion,  affecting  schools  and  libraries 
(or  media  centers).  It  is  notorious  that  these  fifteen  years  (1985-2000)  brought  more  changes  to 
technology  than  those  that  occurred  in  the  past  85  years! 

All  this  means  that  the  conditions  to  move  forward  were  created.  If  powerful  networking  is 
available,  if  the  storage  and  retrieval  of  information  becomes  easy  for  anyone,  if  communicating 
instantaneously  with  any  part  of  the  world  is  possible,  then,  why  not  have  virtual  resource  centers? 

In  the  1980s  the  challenge  of  new  information  technologies  (NIT)  stimulated  many 
contributions  that  made  clear  that  the  future  of  libraries  (or  learning  centers)  depended  on  the  way  these 
iastitutions  were  able  to  deal  with  those  technologies.  A new  concept  of  literacy  is  implicit:  besides 
reading  and  writing,  the  knowledge  of  using  the  computer  is  indispensable  (Breivik,  1985;  Horton, 
1983;  Hubbard,  1987). 

However,  it  is  only  in  the  1990s  that  the  idea  of  a “virtual  library”  (and  “virtual  resource 
center”)  solidifies.  Books  and  journal  articles  were  published  throughout  the  decade  clarifying 
concepts,  discussing  principles  and  proposing  models  (Butler,  1991;  Butterworth,  1992;  Kurzweil, 

1 993 ; Rooks,  1 993 ; Saunders,  1 993 ; Blake,  1 994).  Most  were  cautious,  announcing  the  potential  of  the 
new  format  but  lowering  the  expectancy  levels.  Some  of  them  decided  to  act  as  futurologists  and  we  do 
not  know  whether  they  were  right  or  wrong. 
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The  Situation  in  Portugal 


The  first  great  impact  of  NIT  into  the  Portuguese  school  system  dates  from  the  mid-eighties, 
when  the  Ministry  of  Education  launched  a specific  program  called  MINERVA10.  The  objectives  were 
“to  develop  teaching  about  computers  and  learning  with  computers,  with  adequate  teacher  training” 
(OECD,  1986,  p.  22).  It  had  a modest  start,  considering  the  budget  and  the  number  of  schools  involved 
(about  50  secondary  schools),  but  its  developments  were  much  more  promising.  When  the  project 
ended,  in  1992,  1,172  schools  of  different  levels  of  education  had  participated  in  MINERVA,  more 
than  50,000  teachers  had  attended  in-service  training  and  more  than  100,000  students  had  attended 
classes  or  workshops  regularly  (Ehrmann,  Somekh,  Withers  & Grandbastien,  1994). 

In  addition,  the  teachers  who  were  participants  as  monitors  kept  from  the  project  a great  elan, 
and  most  of  them  completed  university  degrees  in  the  field  and  continue,  in  their  schools  or  elsewhere, 
to  disseminate  what  they  have  learned. 

The  University  of  Minho  had  been  one  of  the  centers  where  MINERVA  had  grown  up. 
Among  the  teaching  staff  from  the  education  field  but  also  from  the  computer  science  field  there 
existed  a strong  feeling  about  the  need  for  developing  continuously  the  pedagogical  approaches  to  NIT 
use.  Therefore,  nobody  was  surprised  when,  in  the  late  eighties  and  the  beginning  of  the  nineties,  the 
University  of  Minho  set  up  several  post-graduate  courses  on  educational  technology,  first  at  master 
level,  then  as  specialization  for  elementary  teachers. 

At  this  time,  elementary  teachers  were  certified  after  a three  year  course  (corresponding  to  a 
bachelor’ s degree).  The  post-graduate  course  consisted  of  two  years  of  study  (thirty  five  credits).  The 
students  who  performed  well  in  a final  exam,  which  consisted  of  the  defense  of  a mini -dissertation,  got 
the  degree  of  “licenciado”,  a degree  that  is  popular  in  Europe  but  does  not  exist  in  the  United  States. 

Those  courses  were  restructured  in  1997,  and  since  then  the  NIT  course  (now  denominated 
“Information  Management  and  Educational  Communication”)  offers  a discipline  of  "Management  of 
Resource  Centers".  This  discipline  aims  to  provide  students  with  the  appropriate  knowledge  and  skills 
to  deal  with  school  media  centers.  As  the  proponent  of  the  inclusion  of  the  discipline  in  the  curriculum, 
one  of  us  (C.  Freitas)  assumed  its  teaching.  His  background  situation  analysis  can  be  synthesized  as 
follows. 

In  Portugal,  elementary  schools  (the  students  enrolled  in  the  course  were  elementary  teachers, 
as  it  has  already  been  said)  are  very  diverse  in  their  formats:  a few  are  big  ones,  mainly  in  important 
towns,  as  Lisbon  or  Oporto,  a few  are  very  tiny,  as  happens  in  rural  regions  (some  schools  have  just 
two  or  three  students!),  and  most  of  them  are  middle  sized.  By  and  large,  first  because  of  the 
immigration  (mainly  in  the  1960s  and  1970s),  then  because  of  the  decreasing  rate  of  births  in  the 
country,  a strong  diminution  of  students  in  schools  became  a reality.  Because  of  that  situation,  new 
legislation  was  enforced  regarding  schools  administration.  Small  schools  were  invited  to  reorganize 
themselves  to  find  partnerships,  building  groups  of  schools  (“agrupamentos”),  that  is,  schools  joined 
under  the  same  administration  in  spite  their  physical  distance,  which  normally  is  not  a substantial  one. 

Each  school  has  its  own  equipment  and  educational  materials;  most  schools  have  small 
libraries.  The  new  administration  logic  challenges  the  way  equipment,  and  learning  resources,  have  to 
be  used.  It  is  not  surprising  that  the  idea  to  join  up  the  different  schools  made  very  clear  the  need  for  a 
new  way  to  make  more  effective  the  existing  resources:  therefore,  little  by  little,  some  resource  centers 
have  been  created. 

It  was  decided  to  introduce  in  the  discipline  some  discussion  about  the  role  the  Internet  plays 
in  the  dissemination  of  information  and  the  emergence  of  true  virtual  resource  centers.  One  of  us  (A. 
Silva),  even  before  the  course  he  was  taking,  was  developing  a web  site  about  his  mother  land,  the 
beautiful  and  historical  Ponte  de  Lima' ',  a town  about  30  kilometers  distant  from  Braga.  He  decided  to 
make  some  arrangements  to  transform  his  site  into  a virtual  resource  center  to  serve  educational 
purposes,  although  it  aims  also  to  reach  tourists  and  other  people  interested  in  the  historical  town. 


10  MINERVA  stands  forMeios  INformaticos  no  Ensino:  Racionaliza^o,  Valorizapao,  Actualiza9ao( Means  of 
INformatics  in  Education:  Rationalization,  Valorisation,  Actualization). 

1 1 Ponte  de  Lima  means  “The  River  Lima  Bridge”.  The  town  still  has  a very  extensive  Roman  bridge,  which  is  one 
of  the  most  beautiful  in  the  country. 
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The  Virtual  Resources  Center  of  Ponte  de  Lima 


A.  Silva,  in  his  final  report  (1999),  evoked  Negroponte.  Negroponte  several  times  drew  our 
attention  to  the  effect  Internet  could  play  in  disseminating  information  (v.g.,  1995,  2000).  The 
development  of  his  idea  comes  first  from  the  passion  for  computers  and  the  Internet. 

The  WWW  allows  teachers  and  students  to  create  their  own  resources  and  learning 
environments.  However,  much  more  important  than  the  act  of  creation  itself  is  the  potential  of  such 
documents  to  become  available  to  many  other  people  - mainly  other  teachers  and  students  - but 
virtually  to  any  human  being  who  uses  the  Internet.  The  information  generated  through  the  Internet  is 
always  accessible,  sometimes  much  more  easy  to  retrieve  than  in  a library,  as  happens  on  consulting  an 
encyclopedia  or  a dictionary  on  line  or  downloading  pictures  and  texts. 

As  it  was  said,  Ponte  de  Lima  is  an  old  town  (population:  c.  2500).  It  is  the  chief-town  of  the 
“concelho”  (the  Portuguese  word  that  stands  for  county,  although  the  dimensions  of  most  “concelhos” 
are  undersized  in  comparison  to  the  U.S.  counties).  The  “concelho”  has  51  “freguesias”  (Portuguese 
word  meaning  a small  village),  with  a total  of  44,000  inhabitants  (Figure  1).  Almost  all  “freguesias” 
have  an  elementary  school  and  some  of  them  have  two;  the  number  of  elementary  schools  is  61 . Ponte 
de  Lima  and  three  other  villages  possess  post-elementary  and  secondary  schools.  Ponte  de  Lima  also 
has  agricultural  schools,  one  at  the  secondary  level  and  other  belonging  to  the  polytechnic  network 
(higher  education).  There  is  at  Ponte  de  Lima  a private  extension  of  a University,  the  Fernando 
Pessoa  University. 
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Figure  l - Map  of  the  " concelho  " de  Ponte  de  Lima 

The  number  of  students  enrolled  ( 1 998-1 999  data)  is  approximately  8,000;  the  number  of 
teachers  is  567.  The  dispersion  of  elementary  schools  isolates  teachers  in  their  schools.  Sometimes, 
they  have  difficulty  in  find  the  materials  they  need  for  their  classes.  Although  each  school  has  a 
minimum  of  resources,  it  is  impossible  to  provide  each  school  with  everything  it  needs.  A virtual 
resource  center  could  help  those  teachers  and  their  students  very  much. 

Since  1998,  following  a deliberation  from  the  national  government,  all  Portuguese  schools 
have  been  equipped  with  the  latest  generation  of  computers  and  on  RDIS  line.  The  first  schools  to  be 
equipped  were  the  secondary  ones;  all  the  elementary  schools  will  have  computers  by  the  end  of  2001. 
This  does  not  mean  that  many  elementary  schools  do  not  have  computers  today;  many  local  authorities 
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anticipated  the  government  deliberation  and  supplied  some  of  the  schools  under  their  jurisdiction  with 
machines. 

This  case,  the  concretization  of  a virtual  resources  center  to  assist  those  schools  made  sense. 
The  design  of  the  center,  however,  also  contemplated  other  kind  of  concerns  A.  Silva  had  in  mind:  as  a 
local  native,  he  could  not  forget  his  fellow-citizens  working  abroad  or  people  interested  in  learning 
more  about  Ponte  de  Lima.  Consequently  the  design  of  the  Web  site  reflected  the  several  publics 
expected  to  visit  it:  teachers,  students,  emigrants,  and  tourists. 

The  visit  to  the  Center  shows  seven  basic  themes: 

Access, 

History, 

Villages, 

Folklore, 

Nature, 

Utilities,  and 
Pictures. 

They  are  accessed  through  a navigation  bar  (Figure  2). 


_AcxijWxs Hi&loria Frcgucaias  Folciore  Nature/a Utilidades Imagens  Navoguu 


Figure  2 - Navigation  bar 

Each  theme  is  divided  into  sub-themes.  The  theme  History,  for  example,  has  the  following 
sub-divisions: 

The  origins  of  Ponte  de  Lima, 

The  river  “Lethes”  myth; 

Characterization  of  the  river; 

The  bi-weekly  market; 

The  “Feiras  Novas”  (Portuguese  words:  New  markets); 

The  “Vaca  das  Cordas”  (Portuguese  words:  the  cow  of  ropes)13; 

Monuments. 

Every  one  of  these  sub-themes  has  plentiful  documents,  written  texts,  pictures,  and  songs. 
Obviously  they  have  links  to  other  themes,  according  to  the  logic  of  the  construction. 

We  can  find  in  this  center  such  varied  items  as  regional  recipes,  a complete  statistical  picture 
of  all  “freguesias”  or  the  timetable  for  buses  and  how  to  call  a cab.  The  quality  of  the  images  is  very 
good,  and  some  images  picture  dramatic  events  of  the  town’s  life  (Figure  3). 

With  these  documents,  teachers  can  easily  pick  out  those  most  relevant  to  their  lessons.  The 
site  has  the  address  http://www.pontcdelima.com.  or  just  http://Dontedelima.com. 

However,  as  some  schools  did  not  have  Internet  access,  200  CD  are  going  to  be  distributed  to 
all  schools  of  the  county,  making  possible  the  utilization  of  the  Center  by  teachers  without  Internet. 
Bearing  in  mind  the  interest  of  potential  tourists,  there  are  envisaged  English  and  French  versions  of 
the  site.  The  problem,  for  the  moment,  is  finding  the  appropriate  financial  support. 

As  a final  requirement  for  the  award  of  the  diploma,  the  jury  who  appreciated  this  work 
congratulated  the  author.  The  Center  has  been  constantly  improved  since  then.  It  began  as  dependent 
on  a large  site;  today  it  is  emancipated,  a “ dot  cow”  site. 

On  the  other  hand,  the  feedback  coming  from  schools  and  teachers  is  very  stimulating.  One 
teacher  who  acted  as  monitor  in  a workshop,  told  us: 


The  fact  that  we  now  have  the  site  www.pontedelima.com  was  a very  important  event  to 
Ponte  de  Lima  ...  it  is  a very  pertinent  tool,  actually  very  useful  for  teachers  and  students  ...  it 
opens  new  perspectives,  because  it  allows  the  utilization  in  the  classroom,  the  teacher  having 
all  information  needed  about  Ponte  de  Lima  . . . [Personally],  during  a workshop  about 
“Introduction  to  the  Internet”,  I found  this  site  was  one  of  the  most  used,  due  to  the  format, 


It  is  a kind  of  bull- fight;  a cow,  not  a bull,  runs  through  the  crowd  secured  by  some  ropes. 
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structure  and  variety  of  effects  the  Author  was  able  to  put  in  it.  It  is  a good  example  of  quality  . 


Figure  3 - A flood  - the  river  Lima  reaches  the  town 's  houses . 

A member  of  the  teaching  staff  of  the  Fernando  Pessoa  University  sent  a short  message: 

Congratulations  on  the  Virtual  Resources  Center  of  Ponte  de  Lima.  Thank  you  for  the 
significant  help  you  gave  us  for  building  our  prospectus  for  foreign  students. 

From  another  point  of  view,  A.  Silva  received  several  e-mails  from  abroad:  the  United  States 
(Massachusetts),  Canada,  United  Kingdom,  France,  Italy,  Brazil,  and  Spain,  among  others.  The  Internet 
is  doing  its  job,  linking  people  in  the  world.  One  of  those  messages  came  from  Poitiers,  France: 

I am  a French  student  (from  a Portuguese  family)  ...  and  I am  preparing  a comparative 
study  of  Portuguese  traditional  tales  and  traditional  tales  from  the  region  where  I live,  Poitou  - 
Charentes  . . . Would  you  tell  me  if  there  are  anthologies  of  tales  from  Ponte  de  Lima  and  its 
environment?  ... 

Therefore,  the  virtual  resource  center  is  working  even  outside  the  region  where  it  was  created. 

Conclusion 

It  seems  clear  that  virtual  resource  centers  have  a place  in  our  school  system,  even  though 
“real”  resource  centers  exist.  Consulting  encyclopedias,  dictionaries  and  databases  around  the  world, 
visiting  museums  and  exploring  endless  opportunities  is  something  that  can  do  much  more  for  children 
than  simple  books  and  traditional  lessons. 

This  experience,  which  we  are  trying  to  monitor  better  next  year,  when  there  will  be 
computers  in  all  classrooms,  challenges  teachers  and  students:  we  think  that  the  time  has  come  to  jump 
from  the  timid  use  of  new  technologies  as  auxiliary  of  the  teaching-learning  process  to  the  creation  of 
stimulating,  imaginative  and  supportive  technological  environments  for  both  teachers  and  learners. 
Virtual  resource  centers  could  be  one  of  the  tools  to  work  out  such  a challenge. 
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Research  Constructs  in  Distance  Education:  Interaction 

Online 


Interaction  has  been  given  number  of  meanings  in  the  context  of  distance  learning.  Computer 
based  training,  which  may  require  nothing  more  than  clicking  on  a button,  has  been  called  interactive, 
as  has  instructional  video  and  systems  that  allow  voting  or  polling.  It  clarifies  matters  if  the  word 
interaction  were  taken  to  only  refer  to  situations  where  a human  response  referred  to  a previous  human 
response  (Mason, ).  This  is  consistent  with  Daniel  and  Marquis’  (1 983)  definition  of  interaction:  “The 
student  is  in  two-way  contact  with  another  person(s)  in  such  as  way  as  to  elicit  from  them  reactions  and 
responses  which  are  specific  to  their  own  reactions  and  responses”  (p.  32),  and  Simpson  and  Galbo’s 
( 1 986)  definition: 

Behavior  in  which  individuals  and  groups  act  upon  each  other.  The  essential  characteristic  is 
reciprocity  in  actions  and  responses  in  an  infinite  variety  of  relationships:  verbal  and  nonverbal.  . 
.Interaction  is  seen  as  a continually  emerging  process,  as  communication  in  its  most  inclusive  sense, 
(p.  38) 

Main  and  Riise  ( 1 995)  conducted  research  to  design  a taxonomy  of  interaction  that  took  into 
account  dimensions  such  as  amount,  timeliness,  method,  and  quality  of  interaction.  Mason  contended 
that  interaction  increases  motivation,  speed  of  assimilation  of  information,  and  length  ofretention  of 
subject  matter  (Mason, ).  When  students  expressed  what  they  had  learned  from  a personal  context  and 
had  offered  differing  points  of  view,  critical  thinking  skills  were  developed  as  learning  became  deeper. 

It  has  been  widely  held  that  interaction  is  critical  to  learning  and  an  essential  part  of  the 
academic  process  (American  Distance  Education  Consortium,  1999;  Berge,  1996;  Hassenplug  & 
Hamish,  1998;  Kearsley,  1990;  Parker,  1999;  Phipps  & Merisotis,  2000;  Shale  & Garrison,  1990b). 
Summers  (1991)  reasoned  that  without  interaction  teaching  was  reduced  to  “passing  on  knowledge  as  if 
it  were  dogmatic  truth”  (p.  14).  This  transferal  model  of  learning  eliminated  any  evaluation  of  the 
information  transferred  (Shale  & Garrison,  1990a).  Parker  (1999)  proposed  that  it  is  not  sufficient  for 
instruction  to  be  transmitted  linearly  from  instructor  to  student  without  interaction.  “Today’s  distance 
education  courses  must  authorize  students  to  question  their  ideas  and  beliefs,  thereby,  encouraging 
provocative  and  interactive  construction  of  personal  knowledge”  (p.  13).  Peters  (1999)  concurred  by 
stating,  “Ifwe  take  distance  education  seriously  and  understand  it  to  be  something  more  than  the  mere 
distribution  of  reading  materials,  we  must  provide  sufficient  opportunities  for  dialogues”  (p.  13). 
Noble  ( 1 999)  emphasized  that  along  with  a low  student-teacher  ratio,  the  one  unquestionable  mark  of 
quality  education  is  that  there  be  significant  interaction  between  two  parties.  Remmers  (1995) 
maintained  that  without  interaction  there  can  be  no  education.  The  concept  of  “no  interaction,  no 
education”  was  used  as  an  online  global  debate  topic  using  a listserv  at  the  University  of  Alberta, 
designed  in  New  Mexico  to  be  used  as  a pre -conference  for  the  1995  International  Council  on  Distance 
Education  in  Birmingham  England,  summarized  at  George  Washington  University  and  Penn  State 
demonstrating  the  potential  of  world-wide  distance  learning.  This  project  produced  an  interaction 
analysis  model  for  examining  social  construction  of  knowledge  in  computer  conferences. 

Years  before  CMC,  Dewey  (1938)  made  an  eloquent  case  for  the  importance  of  interaction  in 
education: 

There  is,  I think,  no  point  in  the  philosophy  of  progressive  education  which  is  sounder  than  its 
emphasis  on  the  importance  of  the  participation  of  the  learner  in  the  formation  of  the  purposes  which 
direct  his  activities  in  the  learning  process,  just  as  there  is  no  defect  in  traditional  education  greater  than 
the  failure  to  secure  the  active  co-operation  of  the  pupil  in  construction  of  the  purposes  involved  in  his 
studying,  (p.  67) 

Moore  (1989)  defined  three  types  of  interaction  found  in  distance  learning:  learner-content 
interaction,  learner-instructor  interaction,  and  learner-learner  interaction.  Moore  considered  learner- 
content  interaction  to  be  the  defining  characteristic  of  education  because  without  content  there  is  no 
education.  This  basic  form  of  interaction  can  be  one  way.  A student  can  interact  with  a book  by 
reading  it  or  with  a videotape  by  watching  it.  Currently,  learner-content  interaction  can  be  seen  in 
computer-assisted  instructional  programs  (CAI)  where  a student  interacts  with  the  contents  of  an 
instructional  CD-ROM  or  program,  which  requires  no  intervention  from  an  instructor. 

Moore  ( 1 989)  considered  learner-instructor  interaction  to  be  highly  desirable.  The  instructor 
motivates  and  stimulates  the  learner  to  enhance  student  interest  and  desire  to  learn.  The  instructor 
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organizes  information  that  is  then  presented  or  demonstrated  to  the  students.  The  teacher  monitors  and 
evaluates  the  students’  mastery  of  the  instruction,  providing  feedback  and  assistance.  Depending  upon 
the  method  of  delivery,  the  student  interacts  with  the  teacher  either  with  a great  degree  of  frequency 
such  as  is  possible  in  two-way  video  or  computer-conferencing,  or  with  a low  degree  of  frequency  in 
the  context  of  a text-based  correspondence  course.  This  form  of  instruction  is  an  individual  dialogue 
between  the  student  and  the  instructor  without  the  benefit  of  a class  of  peers.  Taking  part  in  this  form 
of  instruction  requires  a great  deal  of  autonomy  on  the  part  of  the  learner. 

Learner-learner  interaction  takes  place  between  learners  either  with  or  without  the  benefit  of 
an  instructor.  Moore  (1989)  proposed  that  this  form  of  interaction  could  be  used  to  teach  interaction 
itself  but  noted  that  the  uses  for  it  would  vary  depending  on  the  situation,  ages,  experiences,  and  levels 
of  autonomy  of  the  learners.  Moore  contended  that  learner-learner  interaction  would  challenge  future 
thinking  and  practice. 

Hillman,  Willis,  and  Gunawardena  (1994)  proposed  a fourth  type  of  distance  learning 
interaction:  learner-interface.  This  type  took  the  specific  technologies  used  for  distance  learning  into 
account  adding  another  dimension.  They  proposed  that  the  presence  of  interaction  was  a basic  need  for 
learning  to  occur  and  reinforced  the  importance  of  learner-learner  interaction  when  they  stated, 
“Regardless  of  content,  however,  instructors  should  take  care  to  involve  the  learner  in  actively  using 
the  technology  to  communicate  with  other  learners”  (p.  38). 

In  the  years  since  the  four  types  of  distance  learning  interactions  were  noted,  research  has 
been  conducted  exploring  these  modes  of  interaction.  Powers  and  Mitchell  (1997)  in  a study  of  an 
online  course  observed  that  students  were  able  to  develop  rapport  and  provide  support  for  each  other. 
The  instructor-learner  relationship  changed  to  reflect  the  instructor  becoming  less  of  a dispenser  of 
knowledge  as  a community  of  learners  developed.  Similarly,  Anderson  and  Garrison  (1998)  noted  a 
shift  in  the  balance  of  power  when  interactive  online  components  were  employed,  and  that  a “learning 
community  realized  through  learner-learner  interaction  need  not,  nor  should  not  be  absent  from  the 
distance  education  experience”  (pp.  102-103). 

Harasim  and  Yung  (1 993)  in  discussing  a survey  of  1 76  teachers  and  learners  on  the  Internet 
reported  differences  found  in  CMC  when  compared  with  face-to-face  instruction.  They  discovered  that 
when  online  learner-learner  group  interactions  greatly  increased  the  ensuing  discussions  became  more 
detailed  and  deeper.  It  was  additionally  found  that  personal  communications  between  and  among 
learners  increased.  Bruner  (1 971 ) added  that  much  learning  takes  place  in  a social  context  through  a 
mutual  construction  of  understanding.  Ahem  (1995)  in  studying  computer  conferences  observed  that 
learner-learner  interaction  enhanced  learning  but  also  created  “an  important  social  environment 
wherein  students  engage  their  peers  with  talk  that  informs,  explains,  persuades,  or  even  entertains.  By 
providing  the  opportunity  for  authentic  peer  interaction,  each  student  will  develop  an  awareness  of 
authorship”  (p.  1 34).  During  a year  - long  faculty  seminar  on  distance  learning  held  at  the  University 
of  Illinois,  it  was  concluded  that  “an  online  course  taught  well  creates  a great  deal  of  interaction 
between  the  professor  and  his  or  her  students.  There  must  be  a great  deal  of  interaction  between  the 
students  themselves,  especially  in  wholly  online  courses  for  placebound  students”  (University  of 
Illinois  Faculty  Seminar,  1999,  p.  3 1).  However,  Johnson,  Aragon,  Shalik,  and  Palma  Rivas  (1999) 
argued  against  including  interaction  in  online  courses.  They  proposed  that  online  communication 
lacked  a strong  social  dimension.  The  results  of  their  study  of  student  satisfaction  in  online  and  face- 
to-face  courses  warned  that,  “until  the  technologies  for  online  instruction  better  simulate  real  time 
interaction,  program  developers  need  to  avoid  courses  that  require  frequent  socialization  between  the 
students  and  the  instructor”  (p.  2 1 ). 

Interaction  just  doesn’t  happen.  According  toKearsley  (1990)  extensive  research  insisted  that 
“interactivity  must  be  planned  or  it  is  unlikely  to  occur  (or  be  meaningful).  The  idea  that  interaction 
must  be  explicitly  designed  in  distance  education  seems  a difficult  concept  for  many  instructors  to 
accept  or  understand”  (p.  8).  Anderson  and  Garrison  (1995)  concluded, 

Merely  acquiring  and  using  the  technology,  without  regard  to  the  development  of  opportunities  for 
regular  and  sustained  interaction  between  and  among  users  and  learners,  provided  no  guarantee  that  a 
critical  community  of  learners  would  result.  If  critical  thinking  is  a desired  learning  outcome,  then 
learning  activities  that  capitalize  on  the  interactive  potential  of  the  medium  must  be  planned  and 
developed,  (p.  42) 
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This  paper  proposes  the  use  of  specific  coaching  strategies  to  facilitate  students'  use  of  expert- 
like problem-solving  strategies  while  analyzing  and  solving  instructional  design  case  studies.  Findings 
from  an  exploratory  study,  designed  to  examine  changes  in  students'  problem-solving  skills  as  they 
analyzed  case  studies,  suggested  that  students  could  show  expert  characteristics  at  times,  under  some 
circumstances,  but  did  not  perform  like  experts  on  a regular  basis.  Thirty-seven  students  at  two 
midwestem  universities  analyzed  six  to  ten  case  studies  as  part  of  their  course  assignments.  Both 
quantitative  and  qualitative  data  were  collected;  students'  written  case  analyses  (initial 
conceptualizations  and  recommended  solutions)  served  as  the  primary  data  source.  Comparisons  were 
made  both  within-  and  across-students,  as  well  as  across  time,  to  examine  patterns  and  changes  in 
students' problem-solving  approaches.  Findings  suggested  that  primary  influences  on  the  incidence  of 
expert  performance  seemed  more  external  than  internal  and  might  be  more  aptly  characterized  as 
"coached  expertise.”  Specific  suggestions  are  included  for  coaching  the  development  of  students’ 
problem-solving  skills  within  a case-based  course. 

Professional  educators  face  a daunting  challenge-  preparing  students  to  solve  the  kind  of 
complex,  ambiguous  problems  that  they  will  encounter  as  practicing  professionals.  Unfortunately,  we 
aren’t  always  successful  in  meeting  this  challenge.  For  example,  in  a study  by  Dahlgren  and  Pramling 
(1985),  physicians  noted  the  need  to  reorganize  what  they  learned  in  medical  school  from  a focus  on 
content  areas  to  a focus  on  common  clinical  problems.  In  addition,  engineers  and  business 
administrators  noted  that  the  theories  they  learned  in  school  were  often  too  simplistic  to  help  in  solving 
real-life  problems.  Julian,  Larsen,  and  Kinzie  (1999)  lamented  that  “novice  (instructional)  designers 
frequently  enter  the  workforce  with  an  understanding  of  the  ID  process  but  without  the  knowledge  base 
that  can  help  them  solve  instructional  design  problems  and  develop  solutions”  (p.  2). 

Increasingly,  professional  educators  have  turned  to  the  use  of  cases  in  an  effort  to  help 
students  learn  to  approach  problem  situations  in  the  same  ways  that  practicing  professionals  do.  Case- 
based  instruction  presents  students  with  a re-creation  of  a complex  situation  (a  case)  and  asks  them  to 
analyze  and  solve  the  problems  through  reflection  and  discussion  (Allen,  Otto,  & Hoffman,  in  press). 

In  case-based  instruction  the  learning  focus  shifts  from  the  explicit  knowledge  and  skills  that  form  the 
traditional  academic  curriculum  to  the  development  of  active  knowledge  - what  Whitehead  called 
“wisdom”  ( 1 929).  Active  knowledge  goes  beyond  simply  recalling  information  to  the  ability  to  use  that 
information  to  select  relevant  issues  and  solve  identified  problems.  For  example,  students  in  law  school 
will  read  a case  study  and  participate  in  a dialogue  designed  to  elicit  the  key  issues  and  legal  principles 
in  the  case.  More  than  simply  recalling  information,  the  students  are  asked  to  analyze  the  situation  as  a 
practicing  lawyer  would  (Williams,  1992).  Similar  methods  have  been  used  in  both  business  and 
medical  education  for  nearly  100  years  (Albanese  & Mitchell,  1993;  Christensen,  1987). 

Case-based  instruction  offers  a number  of  advantages  for  professional  education.  Cases  are 
thought  to  be  more  effective  than  didactic  teaching  methods  because  they  (1 ) more  accurately  represent 
the  complexity  and  ambiguity  of  real -life  problems,  (2)  provide  a framework  for  making  explicit  the 
problem-solving  processes  of  both  novice  (student)  and  expert  (instructor),  and  (3)  provide  a means  for 
helping  students  develop  the  kind  of  problem-solving  strategies  that  practicing  professionals  use.  More 
specifically,  case-based  instruction  can  help  students  learn  to: 

• Focus  on  the  big  picture.  Expert  problem-solvers  typically  represent  a problem  in  terms  of 

an  appropriate  underlying  principle  while  novices  tend  to  represent  the  problem  in  terms  of  surface 
features  (Bruer,  1993).  In  case-based  instruction,  knowledge  is  embedded  within  complex  and  ill- 
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structured  problems.  As  a result,  students  have  an  opportunity  to  practice  “spotting”  the  underlying 
issues  and  principles  in  authentic,  relevant  problems. 

• Work  forward  from  wh  at  they  know.  Experts  often  build  on  what  they  know,  generating 
hypotheses  and  looking  for  information  to  test  those  hypotheses  (Johnson,  1988).  In  contrast,  novices 
tend  to  focus  on  what  they  don’t  know,  looking  for  information  to  fill  in  the  gaps.  A case  study,  by 
necessity,  presents  an  abbreviated  view  of  a problem  situation.  Students  are  forced  to  do  the  best  they 
can  with  the  information  that  is  available.  Thus,  they  learn  to  frame  problems  in  ways  that  will  move 
them  forward  toward  a solution. 

• Simultaneously  consider  multiple  factors.  Experts  are  likely  to  consider  the  web  of 
relationships  and  interactions  that  exist  in  the  problem  situation,  while  novices  generally  consider  one 
factor  at  a time  (Perez  & Emery,  1 995).  Cases  allow  students  to  experience  the  “complex  and  dynamic 
forces”  (Rowland,  1992)  that  operate  within  the  kind  of  complex  problems  that  professionals 
commonly  encounter.  Although  individual  students  may  initially  focus  on  a single  factor  when 
analyzing  a case,  case  discussions  tend  to  bring  all  of  these  factors  to  the  table.  This,  then,  facilitates 
consideration  of  all  factors  in  an  interdependent  manner. 

• Generate  tentative  solutions.  Both  experts  and  novices  generate  solution  ideas  early  in  the 
problem-solving  process.  However,  experts  are  more  likely  to  modify  or  eliminate  those  solutions  as 
additional  information  becomes  available  (LeMaistre,  1 998).  Case  studies  tend  to  involve  more  than 
one  key  player,  each  representing  a unique  perspective.  With  practice,  students  can  begin  to  understand 
that  each  player  has  a legitimate  voice.  Students  are  encouraged  to  modify  their  initial  solutions  as 
information  accumulates  and  different  perspectives  are  heard,  including  the  student’s  own  perspective. 

• Consider  potential  consequences  and  implications.  Experts  think  through  their 
recommendations  more  thoroughly  than  novices,  considering  how  those  recommendations  might  be 
implemented  and  what  implications  they  might  have  (Rowland,  1 992).  As  a part  of  the  case  discussion, 
students  are  asked  to  consider  the  consequences  of  their  recommended  solutions  and  to  select  the  one 
with  the  greatest  benefits  and  smallest  risk.  Students  and  instructors  are  encouraged  to  evaluate  each 
other’s  recommendations  and  to  challenge  decisions  based  on  their  assessment  of  the  consequences  for 
all  case  players. 

Case-based  instruction  seems  to  be  a natural  fit  with  professional  education  in  the  field  of 
instructional  design.  Like  other  professions,  instructional  design  is  a problem-solving  enterprise  in 
which  practicing  professionals  combine  creativity  with  technical  skills  to  solve  complex,  ambiguous 
problems.  Also  like  other  professions,  instructional  design  educators  are  looking  to  bridge  the  gap 
between  education  and  practice  (Quinn,  1 994;  Rowland,  Parra,  & Basnet,  1 995)  using  a variety  of 
methods,  including  case-based  instruction  (Ertmer  & Quinn,  1 999).  According  to  Julian  et  al.  ( 1 999), 
case  studies  can  help  instructional  design  students  (1)  draw  connections  between  their  emerging 
knowledge  of  instructional  design  and  the  complex  demands  of  practice,  (2)  reflect  on  relevant  theory 
and  methods  as  they  explore  a greater  number  of  design  issues  in  a broader  array  of  contexts,  and  (3) 
broaden  their  knowledge  base  as  they  collaborate  with  colleagues  to  identify  effective  design  solutions. 

In  order  to  take  advantage  of  the  perceived  strengths  of  case-based  instruction,  the  authors 
developed  an  instructional  design  course  in  which  the  use  of  cases  comprised  the  primary  instructional 
method.  Our  original  purpose  was  research.  We  planned  to  examine  changes  in  students’  problem- 
solving skills  during  a course  in  which  case-based  instruction  was  used.  However,  after  several 
different  offerings  of  the  course,  we  found  that  we  had  learned  more  about  our  use  of  cases  than  about 
changes  in  students’  problem-solving  skills.  That  is,  the  changes  in  our  application  of  case-based 
instruction  appeared  more  substantial  than  changes  in  students’  problem-solving  approaches. 
Furthermore,  as  we  continued  to  modify  and  refine  our  case  discussion  strategies,  we  began  to  identify 
techniques  that  seemed  better  able  than  others  to  facilitate  more  “expert-like”  responses  among  the 
students.  Thus,  we  began  to  consider  ways  in  which  we  might  more  purposefully  “coach”  our  students 
as  they  developed  their  problem-solving  skills. 

The  purpose  of  this  paper,  then,  is  to  describe  our  evolving  use  of  case-based  instruction 
within  an  instructional  design  course.  Recognizing  that  few  guidelines  exist  for  how  to  coach  the 
development  of  students’  problem-solving  skills  using  case-based  instruction,  this  paper  offers  an 
initial  set  of  recommendations,  based  on  data  gathered  during  three  semesters  of  instruction  (five 
different  courses).  The  first  section  of  the  paper  describes  the  context  in  which  case-based  instruction 
was  situated,  including  the  courses  and  the  use  of  cases.  The  second  section  offers  a set  of  guidelines 
for  using  cases  as  an  instructional  method  in  the  education  of  instructional  designers.  Specifically,  we 
present  guidelines  intended  to  help  students  use  and  develop  the  kind  of  problem-solving  strategies 
commonly  used  by  experts. 
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Context 


In  the  spring  of  1998,  the  first  two  authors  co-taught  an  advanced  instructional  design  course 
at  a large  midwestem  university.  One  faculty  member  was  serving  in  a visiting  position  while  on 
sabbatical  leave  from  another  institution.  The  following  two  spring  semesters  (1999  and  2000),  both 
instructors  taught  similarly  structured  courses  at  their  respective  universities.  During  the  spring  2000 
semester,  the  two  courses  shared  a course  Web  site  that  allowed  students  to  participate  in  shared 
electronic  discussions. 

Participants  (n=37)  were  enrolled  in  advanced  graduate  (n=25),  introductory  graduate  (n=7), 
or  advanced  undergraduate  courses  (n=5)  in  instructional  design.  All  but  one  student  volunteered  to 
participate  in  the  study.  Two  undergraduate  students  were  dropped  from  the  study  due  to  incomplete 
data  sets.  Course  assignments  and  procedures  were  the  same  for  participants  and  non-participants. 

Graduate  students  were  enrolled  in  either  an  advanced  instructional  design  course  within  an 
educational  technology  program  or  an  introductory  course  within  a human  resource  development 
program.  Students  in  the  advanced  course  had  taken  0-7  previous  instructional  design  courses. 
Although  an  introductory  instructional  design  course  was  listed  as  a prerequisite,  three  students  were 
admitted  without  the  introductory  course,  having  gained  sufficient  foundational  knowledge  from 
previous  work  experiences.  Graduate  students  ranged  in  age  from  24-51  years,  with  an  average  age  of 
35  years.  Undergraduate  students  were  enrolled  in  the  second  instructional  design  course  within  a 
human  resource  development  program.  Undergraduate  students  ranged  in  age  from  21-24  years,  with 
an  average  age  of  23  years. 

Students  in  all  three  courses  had  a variety  of  background  experiences  including  retail  sales, 
office  work,  classroom  teaching  (junior  high,  high  school,  community  college,  university),  nursing, 
course  development,  engineering  at  a local  manufacturing  plant,  and  training  (formal,  informal,  one- 
on-one  settings).  In  general,  undergraduate  students  had  less  course  work  and  fewer  relevant  job 
experiences  than  the  graduates. 

Procedures 

Students  in  each  course  analyzed  six  to  ten  instructional  design  cases  as  part  of  their  course 
assignments.  Case  studies  were  drawn  from  Ertmer  and  Quinn  ( 1 999)  and  were  used  in  conjunction 
with  other  activities  (guest  speakers,  student  presentations,  project  assignments,  required  readings) 
throughout  the  semester.  There  were  no  assigned  textbooks  in  the  advanced  graduate  course,  although 
additional  readings  were  frequently  recommended  or  required.  The  introductory  graduate  course 
included  an  instructional  design  book  by  Smith  & Ragan  (1999)  and  the  undergraduate  course  used  a 
text  by  Kemp,  Morrison,  and  Ross  (1996). 

Students  in  each  course  also  completed:  ( 1 ) an  open-ended  questionnaire  describing  previous 
experiences  with  instructional  design  and  with  cases,  and  (2)  a self-assessment  survey  indicating  levels 
of  competency  from  1 (weak)  to  5 (strong)  on  sixteen  instructional  design  skills.  The  self-assessment 
survey  was  based  on  the  sixteen  instructional  design  competencies  outlined  by  the  International  Board 
of  Standards  for  Training,  Performance,  and  Instruction  (IBSTPI,  1984).  Students  rated  their  current 
level  of  knowledge  and  skill  on  items  such  as:  assess  the  relevant  characteristics  of  leamers/trainees 
(#3),  develop  performance  measures  (#7),  evaluate  instruction/training  (#11),  and  promote  the  use  of 
instructional  design  (#16).  At  the  end  of  the  semester,  the  self-assessment  surveys  were  readministered 
to  all  participants. 

Students  participated  in  case  discussions  both  in  class  and  on-line.  On-line  discussions  were 
facilitated  via  a listserv  (spring  1999)  or  a Web-based  bulletin  board  (spring  2000)  and  typically 
preceded  the  in-class  discussions.  While  the  listserv  discussions  extended  dialog  among  the  advanced 
ID  students  only,  the  Web-board  discussions  included  students  enrolled  in  both  introductory  and 
advanced  courses  at  the  two  universities.  Various  techniques  were  used  to  initiate  and  facilitate  the 
discussions  including  debate,  role-play,  and  structured  discussion.  Regardless  of  technique,  each 
discussion  revolved  around  one  or  two  basic  tasks:  (1)  analyzing  the  problems  and  issues  in  the  case 
and  (2)  recommending  solutions  for  identified  problems  and  issues. 

Qualitative  analysis  methods  were  used  to  examine  changes  in  students1  case  responses. 
Responses  initially  were  coded  using  the  five  expert  problem-solving  characteristics  (see  pages  4-5), 
but  modified  to  fit  emerging  themes  and  patterns.  For  example,  initial  codes  were  used  to  characterize 
students'  conceptualizations  of  the  case  (reporting  vs.  interpreting),  searches  for  informationfahsent  vs. 
present  information),  attention  to  the  relationship  among  factors  (laundry  list  vs.  coherent  planTlevels 
of  commitment  to  solutions  (dictatorial  vs.  advisory),  and  consideration  of  the  implications  of 
recommendations  (narrow  focus  vs.  broad  focus). 
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These  codes  were  refined  throughout  data  analysis  and  eventually  combined  into  analysis  and 
solution  categories  (see  Appendix  A for  more  detailed  descriptions).  Recognizing  that  problem- finding 
and  problem-solving  are  two  related,  but  different,  skills,  we  examined  students’  analysis  and  solution 
approaches  separately.  We  examined  students'  analysis  approaches  by  looking  at  their 
conceptualizations  of  the  issues,  searches  for  information,  and  attention  to  the  relationships  among 
issues.  We  examined  students’  solution  approaches  by  looking  at  their  attention  to  the  relationships 
among  solutions,  levels  of  commitment  to  proposed  solutions,  and  consideration  of  the 
impact/implications  of  proposed  solutions. 

We  assigned  an  independent  rating  to  each  student’s  analysis  and  solution  response.  If  most  of 
the  components  of  a response  were  novice-like,  the  response  was  rated  as  ’’weak."  If  the  response 
included  a fairly  equal  number  of  novice  and  expert-like  characteristics,  it  was  rated  as  "mixed." 
Finally,  if  most  of  the  components  of  the  response  were  expert-like,  the  response  was  rated  as  "strong." 
This  allowed  us  to  1)  compare,  within-students,  analysis  and  solution  approaches,  2)  identify  patterns 
of  responses,  across-students,  that  were  specific  to  each  case  study,  and  3)  to  identify  patterns  that 
developed  or  changed  across-time. 

Results 

A two-tailed  paired  t-test  (df  = 36)  indicated  a significant  increase  in  students’  ratings  of 
perceived  competency  for  instructional  design  skills  (t  = 8.30;p<  .0001 ) from  thebeginning  to  the  end 
of  the  semester.  Even  though  all  1 6 skills  were  not  specifically  addressed  in  the  case  studies  used  in  the 
courses,  students  judged  that  their  competencies  had  increased  across  skills  (pre-test  mean  = 3.44,  SD  = 
.36;  post-test  mean  = 4. 1 5,  SD  = .2 1 ).  However,  this  increase  in  perceived  competency  was  not 
completely  supported  by  corresponding  changes  in  students’  approaches  to  analyzing  cases  and 
recommending  solutions.  In  general,  students  showed  both  strong  and  weak  responses  throughout  the 
semester.  In  fact,  sometimes  students  demonstrated  a stronger  response  on  an  earlier  case  and  weaker 
response  on  a later  case.  For  example,  when  students’  attention  was  directed  to  the  potential 
implications  of  a solution,  students  were  able  to  consider  the  effects  of  their  recommendations,  but  they 
did  not  always  do  so  on  their  own.  As  another  example,  when  students  were  specifically  asked  to 
classify  one  or  several  issues  in  a case,  their  conceptualizations  tended  to  take  on  a “big  picture” 
approach.  In  contrast,  if  asked  to  simply  describe  the  issues,  students  tended  to  respond  with  more 
surface-level  reporting. 

As  we  began  to  consider  potential  reasons  for  the  uneven  development  and/or  demonstration 
of  students'  problem-solving  skills,  our  research  focus  shifted  from  an  emphasis  on  what  students  could 
or  could  not  do,  to  what  we,  as  the  instructors,  did  or  did  not  do  in  our  role  as  coaches.  In  an  attempt  to 
explain  our  "uneven"  results,  we  reexamined  our  on-line  case  discussions  to  identify  if,  and  how, 
specific  coaching  strategies  may  (or  may  not)  have  supported  students’  performances  relative  to  each 
of  the  five  characteristics  of  expert  problem-solving. 

In  the  section  that  follows  we  describe  the  specific  coaching  strategies  we  used  to  i nitiate  the 
on-line  case  discussions  during  spring,  2000.  Although  we  recognize  that  coaching  can,  and  does, 
occur  throughout  a case  debriefing,  our  discussion  here  focuses  primarily  on  the  case  set-up  as  a critical 
starting  point.  Specifically,  we  examined  the  extent  to  which  two  expert  problem-solving 
characteristics  (conceptualizing  the  issues  in  the  case  and  considering  the  impact  of  recommended 
solutions)  were  facilitated  by  the  use  of  specific  coaching  strategies.  Given  the  preliminary  nature  of 
these  analyses,  we  judged  that  these  two  characteristics  offered  a useful  starting  point.  First  of  all, 
these  categories  were  fairly  well  defined  and,  thus,  made  identification  and  classification  of  students’ 
responses  more  reliable.  In  addition,  the  two  c ategories  comprised  both  an  analysis  and  a solution  type, 
which  we  still  considered  to  be  relatively  distinct  aspects  of  the  problem-solving  process.  In  the  next 
section,  we  describe  students’  responses  to  five  different  case  set-ups:  (1)  structured  discussion,  (2) 
debate,  (3)  reflective  practitioner,  (4)  role  play,  and  (5)  discussion  chain.  Each  technique  is  considered 
in  turn.  (Note:  Throughout  the  following  discussion,  participants  in  the  ID  courses  are  referred  to  as 
"students",  whereas  those  who  are  receiving  instruction  within  the  case  narratives  are  referred  to  as 
’’learners.’’) 

Structured  Discussion 

We  used  a structured  discussion  to  initiate  the  on-line  discussion  for  the  Frank  and  Semra 
case.  In  this  case,  two  U.S. -based  instructional  designers  are  faced  with  the  challenge  of  developing  a 
new  curriculum  to  teach  instructional  design  to  trainers  who  work  for  the  government  of  a southeastern 
Asian  country.  To  begin,  the  instructors  provided  a one-sentence  summary  of  the  case,  followed  by 
three  specific  questions:  (1 ) what  do  you  think  the  learners’  expectations  of  the  new  curriculum  will  be, 
(2)  what  kinds  of  adjustments  do  you  think  the  learners  will  have  to  make,  and  (3)  how  might  you 
facilitate  these  adjustments?  Students  were  not  asked,  specifically,  to  discuss  either  the  underlying 
issues  in  the  case  or  the  impact  of  their  solutions. 
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Conceptualization  of  the  case  issues.  As  the  first  on-line  discussion  of  the  semester,  this 
discussion  starter  seemed  to  lead  to  fairly  structured  responses.  Almost  every  student  (1 6 out  of  1 8) 
answered  each  of  the  three  questions,  one  by  one.  Perhaps  because  this  set  of  questions  seemed  so 
structured,  most  of  the  students  (10  out  of  16)  started  by  reporting  or  summarizing  the  facts  in  the  case. 
However,  even  though  students'  primary  focus  seemed  to  be  on  reporting,  12  students  included  an 
interpretation  of  the  case  events,  either  separately  or  in  conjunction  with  fact  reporting.  [“The  learners 
need  to  see  a need  for  change  before  they  buy  into  a new  system  of  design ."]  ["This  is  similar  to  a 
change  initiative."] 

Interestingly,  our  efforts  to  help  students  broaden  their  initial  conceptualizations  met  with 
little  success.  Questions  we  asked  during  the  discussion  to  explore  the  identified  issues  ["What  is  it 
about  the  culture  issue  that  Frank  and  Semra  should  adjust  to?"]  did  not  generate  any  direct  responses 
from  the  students.  However,  when  another  student  asked  whether  the  case  provided  sufficient  reason 
to  believe  that  a change  should  be  made  in  the  training  methods  being  used  (a  common 
recommendation),  other  students  added  new  interpretations  of  the  issue  ["Who  is  the  client  here?  Who 
has  the  right  to  decide  what  is  or  isn't  appropriate?"].  This  difference  in  students'  responses  to 
instructors  and  peers  may  have  been  related  to  the  timing  of  the  case.  As  the  first  case,  students  may 
have  felt  some  discomfort  with  this  type  of  on-line  environment  or  with  responding  to  the  instructors  as 
participants  in  the  discussion. 

Students'  consideration  of  the  impact  of  recommendations.  The  three  questions  that  opened  the 
discussion  asked  students  to  make  recommendations  but  did  not  ask  them  to  explain  the  impact  or 
implications  of  those  recommendations.  Additionally,  the  instructors  asked  few  follow-up  questions 
during  the  discussion  and  those  that  were  asked  generally  focused  on  recommendations  rather  than 
implications  [“Any  thoughts  about  what  Frank  and  Semra  might  do  to  make  sure  that  it  goes  well  with 
these  trainers?"].  In  retrospect,  there  were  opportunities  during  the  discussion  to  ask  the  students  to 
explore  implications  (perhaps  using  questions  such  as  "What  effect  might  that  have  on  the  current 
trainers?")  but,  unfortunately,  we  failed  to  take  advantage  of  these  opportunities. 

Predictably,  students  provided  recommendations,  often  stated  in  specific  terms  [“I  should 
convince  them  that  the  system  approach  is  effective.”]  [“  I will  look  for  new  or  young  trainers.”]. 
However,  there  was  little  consideration  of  the  implications  of  those  recommendations.  When  students 
did  consider  implications,  they  were  typically  limited  to  a narrow  focus  on  how  a recommendation 
would  solve  the  identified  problem  [“Once  the  interactive  atmosphere  is  established,  the  losing  face 
problem  will  not  exist  anymore.”].  Little  apparent  thought  was  given  to  the  broader  impact  of  a 
recommendation,  such  as  how  it  might  affect  other  participants  within  the  case  or  fit  with  other 
recommendations. 


For  the  second  case  we  facilitated  an  on-line  debate.  The  Denny  Clifford  case  is  about  a 
design  consultant,  with  a traditional  instructional  systems  background,  who  is  hired  by  a science 
educator  to  create  constructivist  learning  materials.  As  part  of  the  set-up  for  this  discussion,  the 
instructors  provided  the  following  conceptualization:  "This  case  raises  some  important  questions  about 
working  with  clients  who  view  things  quite  differently  from  us."  Students  were  then  asked  to  argue 
whether  or  not  an  instructional  designer  should  accept  such  an  assignment.  Students  were  assigned  to 
one  of  the  two  positions  and  encouraged  to  engage  in  some  "friendly  competition." 

Conceptualization  of  the,  case  issues.  In  response  to  this  set-up,  almost  every  posting 
represented  a conceptualization  of  why  Denny  should  or  should  not  take  this  job.  Students  were  forced 
to  support  their  positions,  which  led  to  a variety  of  sub-conceptualizations  of  the  issue.  Some  students 
argued  in  terms  of  "comfort'1  or  "stress"  issues;  others  argued  in  terms  of  "risk,"  "challenge,"  or 
"growth  possibilities."  Because  of  this  set-up,  few  students  felt  the  need  to  cite  facts  from  the  case. 
Almost  every  student  interpreted  the  case  from  the  point  of  view  they  were  assigned.  The  debate-style 
set-up  seemed  to  be  an  effective  way  to  get  students  to  think  beyond  the  facts  of  the  case  to  the  positive 
and  negative  sides  of  an  issue.  In  addition,  by  providing  a primary  issue  to  chew  on,  the  instructors 
modeled  how  to  conceptualize  issues  in  a case. 

Students'  consideration  of  the  impact  of  recommendations . The  debate  format  appeared 
somewhat  successful  in  helping  students  focus  on  implications,  especially  among  students  who  argued 
the  “yes”  position.  Students  explored  implications  from  both  a present  and  future  perspective.  In  terms 
of  present  implications,  some  students  noted  that  accepting  the  assignment  would  require  additional 
work  for  both  designer  and  client  [“He  may  need  to  do  a little  extra  work.”]  [“She  may  have  to  play  a 
larger  role  in  the  process  . . .”].  Other  students  included  suggestions  for  increasing  the  likelihood  of 
success  on  the  assignment  [“Perhaps  Denny  needs  to  talk  to  the  people  using  Oakes’  methods  and  find 
out  how  they  are  actually  using  them.”]  [“First,  he  can  begin  by  identifying  the  aspects  that  he  has  in 
common  with  Oakes  . . .”].  In  terms  of  future  implications,  some  noted  that  accepting  the  assignment 
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would  help  the  designer  develop  his  career  [“Working  with  Oakes  , , , could  open  up  even  more  doors 
for  him  to  design  for  others  and  get  him  more  jobs,”]  [“If  he  begins  to  limit  his  range  of  content  and 
philosophies  as  a designer,  he  will  be  compromising  his  career.”]. 

Reflective  Practitioner 

A reflective  practitioner  set-up  was  used  with  the  Michelle  Nguyen  case.  Michelle  Nguyen  is 
a designer/programmer  working  for  a large  medical  center.  She  encounters  a number  of  surprises  as  s he 
prepares  to  roll  out  a new  product,  carefully  designed  to  reflect  the  requirements  of  a front-end  analysis 
completed  nine  months  earlier.  In  this  type  of  case  set-up,  students  were  asked  to  assume  the  role  of 
the  instructional  designer  in  the  case,  and,  as  a reflective  practitioner,  consider  "lessons  learned"  from 
the  specific  experience.  Rather  than  place  blame  on  various  stakeholders,  students  were  encouraged  to 
"move  beyond"  simply  describing  what  happened  to  explaining  at  least  one  thing  that  would  be 
important  to  do,  or  not  do,  the  next  time. 

Conceptualization  of  the  case  issues.  A reflective  practitioner  set-up  forced  students  to 
prioritize  the  important  lessons  learned  from  the  case.  This  took  them  beyond  facts  to  their  own 
interpretation  of  the  important  issues.  For  example,  some  of  the  lessons  learned  included  1)  the  need 
for  continual  communication,  2)  revisiting  steps  in  the  ID  process  as  you  go  along,  and  3)  having  a 
backup  plan  for  dealing  with  problems.  The  lessons-leamed  approach  seemed  to  help  students 
conceptualize  the  primary  issues  in  the  case  in  terms  of  important  design  processes  and  procedures. 

Consideration  of  the  impact  of  recommendations.  The  students  explored  the  implications  of 
their  original  “lessons”  more  in  this  discussion  than  in  previous  discussions.  In  addition,  the  specificity 
of  the  instructors'  follow-up  questions  seemed  to  have  an  effect  on  the  discussion  that  followed.  For 
example,  relatively  general  questions  [“How  do  you  think  a focus  group  might  have  been  helpful  in  this 
case?”]  resulted  in  few  responses  from  the  students.  In  contrast,  questions  that  were  more  specific 
[“But  since  a learner  analysis  had  already  been  done,  wouldn’t  this  slow  down  progress  on  the  project . 

, .?”]  resulted  in  a discussion  that  involved  as  many  as  twelve  different  responses  from  the  students. 

Rale  Elay 

In  two  different  case  discussions,  students  were  assigned  roles  to  play  and  asked  to  view  the 
case  from  that  person’s  perspective.  In  the  Haley  Lawrence  case,  students  were  assigned  a role  and 
asked  to  “get  inside  that  person’s  head”  and  identify  the  issues  and  concerns  of  that  one  person.  Haley 
Lawrence  is  a case  involving  a variety  of  stakeholders  within  a client  organization.  Although  Haley  is 
hired  to  help  the  organization  develop  training  materials  for  their  sales  representatives,  she  encounters 
several  people  in  the  organization  who  appear  uncertain  about  her  role  and  who  find  her  presence 
threatening. 

The  Andrew  Stewart  case  deals  with  the  complexities  and  subtleties  of  integrating  the 
contributions  of  different  groups  of  stakeholders  involved  with  a large  design  project.  A secondary 
focus  is  on  planning  and  completing  a formative  evaluation  for  a product  that  is  still  being  developed. 
The  case  ends  with  the  team  in  "crisis  mode"  as  a deadline  looms  and  the  information  collected,  to- 
date,  is  unacceptable.  In  the  Andrew  Stewart  case,  students  were  assigned  a role  and  asked  to  consider 
what  they  would  do  differently  if  they  could  start  over  and  what  they  would  do  now,  given  the  situation 
as  it  is. 

Students  were  more  active  in  these  two  discussions,  possibly  because  they  had  participated  in 
several  previous  case  discussions,  but  perhaps  also  because  of  the  safety  of  playing  a role.  Students 
could  express  themselves  more  directly  and  emphatically  because  they  were  presumably  expressing  the 
views  of  someone  else  (a  character  in  the  case).  One  result  is  that  both  discussions  included  virtual 
arguments  between  role  players.  [“To  be  honest  with  you,  Mr.  Sumida,  you’re  changing  the  work 
contract  on  me.”  “Ms.  Lawrence,  you  were  brought  in  the  picture  to  solve  a problem  not  to  create  new 
ones.”]  [“It  was  not  me  who  was  trying  to  create  new  tasks.  Instead  I was  trying  my  best  to  focus  my 
tasks  on  our  original  contract.”]  Students  tended  to  stay  in  character  throughout  the  discussion,  often 
responding  to  one  another  in  the  first  person.  [“Why  should  these  be  my  problems?”]  [“Don’t  put  me  in 
a bind.”] 

Conceptualization  of  the  case  issues.  Despite  an  involved  and  intense  on-line  discussion,  the 
Haley  Lawrence  role-play  appeared  only  mildly  successful  at  helping  students  frame  and  address 
design  issues  in  the  case.  While  students  may  have  considered  how  one  individual  perceived  the  issues 
in  the  case,  they  did  not  work  very  hard,  if  at  all,  to  understand  each  other's  issues.  Students  enjoyed 
playing  personalities,  as  noted  earlier,  but  did  not  seem  to  move  outside  the  personalities  to  consider 
broader  case  issues.  However,  in  the  students'  defense,  they  participated  in  this  discussion  exactly  as 
they  were  instructed.  Had  we  encouraged  them,  sometime  during  the  discussion,  to  switch  hats  with 
another  stakeholder,  we  might  have  been  more  successful  in  moving  beyond  personalities  to  issues. 

The  second  role-play  set-up  was  primarily  solution-oriented.  Although  students  were 
assigned  roles  to  play,  they  were  told,  right  from  the  beginning,  that  they  all  shared  responsibility  for 
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the  current  situation.  Thus,  students  refrained  from  blaming  others,  and  looked  for  ways  to  help 
salvage  the  situation.  As  the  role  players  considered  their  own  responsibilities  in  causing  the  situation, 
conceptualizations  emerged  ["I  should  not  have  involved  a group  that  wasn't  as  motivated  about  this 
project  as  the  others."]  [”I  think  that  the  major  flaw  in  what  happened  had  to  do  with  communication."] 
["I  should  have  been  more  professional."]. 

Consideialign  of  the  impact  of  recommendations.  Students  did  consider  impact  and 
implications  in  these  discussions.  On  occasion,  this  included  the  practicalities  of  implementing 
suggestions  that  were  made.  [“You  can  have  the  St.  Louis  team  send  back  the  manuals  on  a weekly 
basis.”]  Typically,  however,  this  was  limited  to  a narrow  view  of  how  suggestions  made  by  others 
affected  their  own  roles.  [“Why  should  these  be  my  problems?”]  [“If  you  could  show  me  a little  respect 
. . .”]  Students  seemed  to  have  difficulty  stepping  outside  of  their  assigned  roles  to  see  the  situation 
more  broadly  or  to  acknowledge  the  perspectives  of  the  other  role  players. 

Piscussion  Chain 

For  the  Sandra  Hernandez  case,  students  were  asked  to  structure  their  discussion  in  the  form 
of  a chain  reaction.  The  Sandra  Hernandez  case  describes  how  an  instructional  designer,  Jake 
Spaulding,  was  asked  to  help  Sandra  "fix"  problems  she  was  having  with  her  freshmai  engineering  lab. 

In  order  to  help  Sandra,  Jake  must  balance  a number  of  contradictory  requirements  and  work  within 
numerous  resource  constraints.  For  this  discussion,  one  of  the  instructors  started  by  describing  what  he 
saw  as  the  most  important  issue  and  then  proposed  a solution.  Students  then  were  instructed  to  either 
agree  or  disagree  with  a posted  comment.  If  they  agreed,  they  had  to  add  something  to  the  comment;  if 
they  disagreed  they  had  to  pose  a counter  argument. 

Conceptualization  of  the  case  issues.  This  type  of  set  -up  provided  students  with  a model  of 
how  to  conceptualize  one  issue  in  the  case  and  then  pose  a relevant  solution.  By  requiring  students  to 
add  to  each  comment  that  they  responded  to,  we  forced  them  to  consider  other  important  issues  that 
had  been  ignored  or  potential  consequences  of  proposed  solutions  that  had  not  been  considered. 
Students  looked  for  nuances  in  interpretation  so  that  they  could  enhance  or  offer  an  alternative  to  a 
comment  already  posted.  ["I  don't  agree  that  time  is  the  main  issue.  I believe  the  main  issue  is  what 
will  be  the  best  approach  to  teach  the  class. ..I'll  consider  time  as  one  of  many  factors  that  are  involved 
in  the  main  issue."]  Thus,  conceptualizations  were  more  complex  than  in  previous  cases,  and  evidence 
was  typically  provided  to  support  specific  conceptualizations.  It  is  important  to  remember  that  this 
discussion  occurred  late  in  the  semester,  which  may  partially  explain  the  increased  complexity  of 
students' responses. 

Consideration  of  the  impact  of  recommendations.  The  “ground  rules”  inherent  in  this  type  of 
set-up  encouraged  the  students  to  think  more  directly  about  the  implications  of  posted  solutions.  They 
could  agree  or  disagree  with  a recommendation,  but  only  after  giving  it  some  thought  - what  benefits 
or  limitations  did  they  see.  One  result  was  that  students  seemed  to  take  a relatively  broad  view  of  a 
recommended  solution.  Sometimes  this  took  the  form  of  adding  suggestions  about  how  to  implement  a 
solution.  For  example,  one  recommendation  was  to  have  the  lab  already  set  up  for  the  learners. 
Students  added  their  thoughts  about  how  to  make  this  work.  [“I  can  see  this  happening  with  the  classes 
being  divided  into  two  sessions.”]  Sometimes  it  took  the  form  of  pulling  in  other  issues.  For  example, 
another  recommendation  was  to  conduct  the  lab  activities  via  computer  simulation.  Students  discussed 
this  solution  in  terms  of  cost  [“Although  this  seems  like  a good  solution,  it’s  expensive  and  time- 
consuming.”]  [“If  you  cost  out  80  students  per  semester,  it  would  not  take  long  to  make  this  program 
cost-effective.”],  access  [“Are  there  computers  available  for  all  of  the  students  to  have  access?”],  and 
instructional  methods  [“From  the  case,  it  seems  to  me  that  the  experiments  are  a very  ‘hands  on’  type 
of  procedure.  I think  computer  simulation  is  not  good  enough  for  this  type  of  experience.”].  In  order  to 
agree  or  disagree  with  another  person's  comment,  students  had  to  carefully  consider  the  many 
implications  of  a recommended  solution.  This,  then,  led  to  a fairly  sophisticated  understanding  of  the 
impacts  of  any  single  solution. 

Coaching  Guidelines 

In  general,  students  in  our  case-based  courses  perceived  themselves  as  more  competent  at  the 
end  of  the  semester  than  at  the  beginning.  However,  the  problem-solving  strategies  they  used  did  not 
show  the  same  kind  of  improvement.  Students  showed  both  strong  and  weak  responses  to  cases 
throughout  the  semester,  suggesting  that  their  problem-solving  expertise  was  not  internalized.  They 
could  show  characteristics  of  expert  problem-solvers  at  times,  under  certain  circumstances,  but  did  not 
do  so  on  a regular  basis.  The  primary  influences  on  their  analysis  and  solution  responses  appeared  more 
external  than  internal. 

One  of  the  external  influences  on  students'  responses  was  the  way  the  case  discussion  was  set 
up  by  the  instructors.  In  the  previous  section,  we  described  the  coaching  strategies  we  used  to  initiate 
on-line  case  discussions  and  the  results  that  followed  their  use.  Specifically,  we  related  these  results  to 
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two  expert  problem-solving  strategies:  ( 1 ) the  ability  to  conceptualize  situations  in  terms  of  underlying 
principles  rather  than  surface  details,  and  (2)  the  ability  to  consider  the  impact  and  implications  of 
recommended  solutions  in  broad,  rather  than  narrow,  terms.  In  this  section,  we  abstract  from  these 
results  several  strategies  that  instructors  can  use  to  guide  their  own  coaching  efforts.  We  have 
purposely  described  the  strategies  in  broad  terms  in  order  io  present  strategies  that  can  be  applied 
through  various  techniques  to  cases  in  a variety  of  content  areas.  Where  possible,  we  illustrate  the 
strategies  with  examples  from  our  case-base  courses. 

Coaching  Students1  Conceptualization  of  Case  Issues 

Strategy  1 : Structure  the  discussion  bv  giving  students  an  initial  role  to  plav  or  a position  to 
take  in  the  discussion.  In  our  case  discussions  we  used  debate,  role-play,  and  reflective  practitioner 
formats,  all  of  which  worked  reasonably  well  as  a way  of  implementing  this  strategy.  Besides 
supporting  students’  conceptualization  of  the  case  issues,  several  other  benefits  appeared  to  result  from 
using  this  strategy: 

Safety.  Students  can  present  observations  and  opinions  within  the  context  of  the  assigned  role, 
and  not  feel  like  they  are  exposing  their  own  views  to  disagreement,  ridicule,  or  evaluation.  For 
example,  during  the  on-line  discussion  of  the  Haley  Lawrence  case  we  asked  students  to  play  one  of 
five  characters  in  the  case  and  to  describe  how  that  person  viewed  the  case  situation  including  his/her 
issues  and  motives.  Students  used  a variety  of  creative  approaches  to  express  the  thoughts,  feelings, 
and  opinions  of  their  assigned  character.  One  student  wrote  in  a stream  of  consciousness,  which 
enabled  other  students  to  read  this  character’s  innermost  feelings  and  thoughts  about  the  case.  [“I  can’t 
believe  Sumida  didn’t  tell  me  about  his  task  force  and  this  list.  Great,  I see  where  this  is  going... When 
is  Califano  coming  on  board  again?  I can’t  wait  until  then,  okay  schedule  an  appointment  with 
Katz. . .”]  Another  student  summarized  her  response  in  the  form  of  a memo  to  another  character.  A third 
student  set  the  scene  and  then  began  sharing  his  thoughts  on  the  case  by  talking  to  himself  [“Sumida 
was  smoking  his  last  cigarette  before  going  to  bed,  while  reviewing  a chapter  of  Covey’s  Seven  Habits 
of  Highly  Effective  People.  He  flipped  over  the  open  book  on  the  table  and  he  paused  into  thinking 
about  his  last  meeting.  The  meeting  scene  was  in  front  of  his  eyes,  he  started  talking  to  himself  and 
creating  his  own  scenarios  about  the  new  project.”].  The  safety  of  responding  from  the  perspective  of 
an  assigned  role  appeared  to  give  students  permission  to  respond  both  boldly  and  creatively,  without 
fear  of  being  criticized  by  their  peers. 

^lanm^point-  The  assigned  role  or  position  gives  students  some  initial  guidance  and/or 
parameters  that  help  them  form  their  initial  response.  For  example,  when  we  began  discussing  the 
Denny  Clifford  case  we  asked  students  a single  question:  “The  first  question  that  needs  to  be  addressed 
is  whether  it’s  even  possible  to  design  instruction  for  someone  whose  philosophy  is  diametrically 
opposed  to  our  own  - 2nd.  when  we  don’t  really  understand  that  other  philosophy.”  We  then  asked  half 
of  the  students  to  “present  reasons  why  Denny  should  ml  try  to  work  with  Cynthia”  and  the  remaining 
students  to  “present  reasons  why  Denny  should  be  successful  working  with  a client  with  a different 
philosophy.”  The  assigned  position  enabled  students  to  engage  in  a lively  debate  as  they  argued  both 
why  Denny  should  take  the  job  [“My  opinion  is  that  Denny  Clifford  should  ‘say  yes  bravely’  to  taking 
this  job  especially  because  he  should  remind  himself  that  he  is  an  independent  consultant  for  a reason  . 

. . that  he  must  try  to  continually  learn  different  philosophies  of  education  so  that  he  is  able  to  innovate 
more  instructional  designs”]  and  why  he  should  not  take  the  job  [“My  position  is  that  Denny  Clifford 
should  just  say  no  to  taking  this  job.  He  reminds  himself  that  he  is  an  independent  consultant  for  a 
reason  . . . that  he  can  choose  the  jobs  he  wants  to  work  on.”].  In  addition,  being  assigned  a position 
that  one  didn't  necessarily  agree  with  encouraged  students  to  consider  strong  arguments  from  both  sides 
of  the  issue.  This,  then,  provided  them  with  a more  complete  view  of  the  situation,  allowing  them  to 
see  that  there  were  advantages  and  disadvantages  to  either  decision  the  designer  might  make. 

Multiple  perspectives.  Assigning  different  roles  within  a single  discussion  helps  ensure  that 
the  viewpoints  of  different  key  players  within  the  case  will  be  voiced.  This  results  in  a broader  view  of 
the  case.  Following  this,  additional  questions  can  be  posed  to  compare  and  contrast  the 
conceptualizations  or  interpretations  of  the  case  represented  by  the  multiple  roles.  Returning  to  the 
Haley  Lawrence  case,  each  student  played  the  role  of  one  of  the  five  characters  in  the  case,  which 
meant  that  several  students  were  playing  the  same  role  during  the  discussion.  Therefore,  each  character 
in  the  case  had  at  least  one  voice  in  the  discussion.  During  the  discussion,  the  instructors  can  also  help 
students  move  beyond  their  own  role  to  see  the  viewpoints  of  the  other  characters  [“So  what  exactly  do 
you  think  is  motivating  his  actions?  And  how  can  Lawrence  get  through  to  him  and  everyone  else)?”]. 
However,  this  may  need  to  take  the  form  of  explicit  directions,  as  the  students  in  our  courses  did  not 
seem  inclined  to  do  this  on  their  own. 
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Strategy  2:  Begin  the  discussion  with  a structure,  but  avoid  rigid  adherence  to  that  structure. 
It’s  important  to  remember  that  the  instructor  has  two  basic  tasks  in  the  case  discussion:  ( 1 ) setting  up 
the  discussion  to  create  a dialog  among  the  students  and  (2)  facilitating  the  discussion  to  keep  the 
dialog  going.  A flexible  discussion  structure  helps  with  both  of  these  tasks. 

Creating  a dialog  The  case  set-up  helps  to  “prime  the  pump”  by  providing  students  with  an 
initial  shared  framework  for  thinking  about  the  case  and  expressing  their  thoughts.  For  example,  in  the 
chain-reaction  approach  used  for  the  Sandra  Hernandez  case  we  started  the  chain  by  (1)  making  a 
statement,  (2)  asking  an  open-ended  question,  and  (3)  presenting  a set  of  ground  rules  for  the 
subsequent  discussion.  Specifically,  one  instructor  began  the  chain:  1)  “I  think  the  biggest  problem 
here  is  the  students’  backgrounds.  They  just  aren’t  ready  for  this  course.  To  solve  this  problem,  I think 
Jake  should  find  a way  to  simplify  the  content  of  the  course  so  that  it  is  more  in  line  with  the  academic 
background  of  the  students  coming  into  the  course.  2)  What  do  you  think?  3)  If  you  agree,  you  must 
add  to  the  posted  comment  in  some  way.  If  you  disagree,  you  must  offer  an  alternative  of  some  kind.” 
By  including  both  guidelines  and  an  example  of  how  to  follow  the  guidelines,  we  jump-started  the 
discussion  and  made  it  easy  for  others  to  jump  in. 

Maintaining  the  dialog  Once  the  discussion  starts  and  the  ideas  are  flowing,  the  initial 
structure  has  served  its  purpose.  In  fact,  students  sometimes  find  it  more  constraining  than  facilitating. 
The  i nstructor’s  tasks  at  this  point  are  to  help  the  students  find  the  connections  among  the  points  being 
made  and  relate  their  discussion  to  the  “big  picture”  issues  in  the  case.  For  example,  during  the 
discussion  of  the  Denny  Clifford  case,  one  student  suggested  an  analogy  to  support  her  point.  There 
were  different  interpretations  of  the  analogy,  which  lead  to  a digression  in  the  discussion.  One  of  the 
instructors’s  maintained  and  refocused  the  dialog  by  looking  beyond  the  analogy  and  posting  a 
question.  [“Christie’s  comment  brings  up  ...  two  good  questions:  (1)  Do  you  have  to  understand  a 
philosophy  in  order  to  design  instruction  to  teach  that  philosophy;  (2)  Do  you  have  to  subscribe  to  the 
philosophy  in  order  to  design  instruction  to  teach  that  philosophy?].  In  maintaining  the  flow  of  the 
discussion,  it's  important  to  remember  that  we  want  students  to  do  more  than  just  'talk';  our  primary 
goal  is  to  maintain  a focused  discussion  that  continues  to  address  the  issues  in  the  case. 

Coaching  Students'  Consideration  of  Implications  of  Solutions 

Strategy  1 ; Ask  specific  questions  and  limit  the  number  that  you  ask  During  the  initial  set-up, 
it  is  possible  to  ask  students  to  identify  a problem,  suggest  a solution,  and  describe  the  possible  impact 
of  that  solution.  However,  this  may  be  too  much  for  the  initial  set-up.  It  may  be  easier  and  better  to 
explore  these  separately  as  the  discussion  progresses.  When  the  initial  set-up  is  too  restrictive  students 
do  not  have  room  to  express  their  views  and  may  feel  that  they  are  providing  “answers”  rather  than 
discussing  ideas.  Once  the  discussion  is  going,  however,  students  can  more  readily  respond  to  specific 
questions  about  the  impact  of  their  recommendations  on  case  stakeholders.  Benefits  of  this  strategy 
include: 


Prevents  students  from  being  overwhelmed.  A multi-part  initial  question  may  overwhelm 
students,  especially  if  it  appears  to  require  specific  answers.  For  example,  we  began  the  discussion  of 
the  Frank  and  Semra  case  with  three  specific  questions,  described  above.  The  students  were  slow  to 
respond  to  these  questions,  reporting  that  they  felt  overwhelmed  by  the  number  of  questions  and  that 
they  were  concerned  about  posting  “correct  answers.”  In  contrast,  we  began  the  discussion  of  the 
Andrew  Stewart  case  by  asking  the  students  to  discuss  two  questions  from  their  character’s 
perspective:  ( 1 ) If  you  had  to  do  it  over  again,  what  would  you  do  differently?  (2)  Now  that  you’re  in 
the  pickle  you  are  in,  what  will  you  do  next?  This  technique  enabled  students  to  jump  into  the 
discussion  without  feeling  like  they  were  overwhelmed  or  had  to  come  up  with  the  right  answer.  A 
result  was  that  students  joined  the  discussion  more  quickly  and  freely  [“I  obviously  should  not  have 
tried  to  involve  a group  that  wasn’t  as  motivated  about  this  project  as  the  other  participants. ..”]  [“...If  I 
had  to  do  it  over  again,  I suppose  I should  have  reacted  to  my  gut  instinct  when  I saw  the  St.  Louis 
team  making  faces.  I’d  forgotten  how  some  designers  think  they  know  it  all  as  soon  as  they  land  a nice 
paying  job  right  out  of  school. . .”].  It  is  important  to  pay  close  attention  to  the  manner  in  which  a case 
discussion  is  opened;  concern  over  "correct  answers"  is  likely  to  stifle  participation. 

Creates  open  dialog  among  students.  Asking  specific  questions  encourages  students  to  1 ) 
respond  to  one  another’s  problem  identification,  2)  suggest  a solution  to  a problem  described  by 
someone  else,  or  3)  describe  the  impact/implications  of  a recommendation  made  by  someone  else.  For 
example,  throughout  the  Sandra  Hernandez  case  students  actively  responded  to  one  another’s  answers 
using  the  “chain-reaction”  approach  and  offered  alternative  perspectives  on  the  impact  and  implications 
of  the  case.  Students  often  responded  directly  to  a person  and  posed  additional  questions  [“Your 
consideration  about  the  limited  budget  available  is  nice,  but  how  could  students  ‘collect  data’  without 
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conducting  real  experiments  or  computer  simulations?”]  [“I  still  disagree.  How  will  the  students’ 
conduct  the  experiments  if  the  equipment  isn’t  set  up?  Will  the  students  still  set  it  up?  If  they  do  then 
the  diagrams  may  use  even  more  valuable  time  when  the  students  study  them.”].  Having  permission  to 
disagree  with  each  other's  interpretations  allowed  for  a rapid  exchange  of  comments,  each  providing  a 
new  consideration  of  possible  impacts. 

Strategy  2;  Look  for  opportunities  to  join  the  discussipn.  but  participate  carefully.  As  the 
discussion  progresses,  the  instructor’s  primary  responsibility  is  to  maintain  the  dialog.  To  do  this,  it’s 
important  to  continually  look  for  opportunities  to  add  questions  and  comments  designed  to  help 
students  see  connections  - to  other  students  and  to  the  big  ideas  that  frame  the  case.  At  the  same  time, 
it’s  important  to  monitor  the  possible  effects  that  your  input  may  have  on  students.  Students  sometimes 
perceive  comments  from  the  instructor  as  the  “answer,”  which  may  impede,  rather  than  support,  the 
dialog.  In  these  situations,  it  is  often  better  to  resist  the  natural  tendency  to  add  your  own,  more  expert 
views  and  let  the  students  come  to  their  own  understanding  of  a case  and  its  solution.  Of  course,  this 
doesn’t  mean  that  instructors  should  avoid  participating  in  the  discussion.  It  means  only  that  instructors 
should  be  constantly  aware  of  the  potential  effects  that  their  participation  may  have.  Benefits  of  this 
strategy  include: 

Models  expert  responses.  When  the  instructor  models  expert  responses  the  students  get  a 
clearer  picture  of  what  is  required  of  them.  For  example,  in  response  to  an  ongoing  discussion  of  the 
Michelle  Nguyen  case,  one  instructor  wrote:  “Playing  devil’s  advocate,  even  if  Michelle  had  conducted 
monthly  (or  quarterly)  meetings  with  Alex  and  Susan,  she  still  may  not  have  found  out  soon  enough 
about  the  library  hours  not  being  sufficient,  I mean  when  is  soon  enough  when  you’re  creating  CBI? 
And  maybe  the  library  hours  aren’t  the  real  issue....  So  the  question  I have  is-  do  you  think  this  case 
primarily  illustrates  a front-end  analysis  problem  or  an  implementation  problem?  Ideas,  anyone?”  This 
comment  was  an  example  of  both  considering  the  implications  of  a recommendation  that  had  been 
made  (monthly  meetings)  and  looking  for  the  underlying  principle  (Do  you  think  this  case  primarily 
illustrates  a front-end  analysis  problem  or  an  implementation  problem?).  Also,  by  prefacing  the 
comment  with  the  warning,  "playing  devil's  advocate,"  the  students  realize  that  their  previous 
comments  are  not  being  criticized.  In  this  way,  students  are  encouraged  to  think  about  the  issues  in 
ways  they  previously  hadn't  considered. 

Supports  ongoing  dialog  The  instructor’s  comments  and  questions  can  also  encourage 
students  to  elaborate  their  thoughts  about  a point  raised  in  the  discussion.  Other  students  can,  then, 
respond  to  the  elaboration,  thereby  continuing  the  dialog.  For  example,  during  the  discussion  of  the 
Haley  Lawrence  case,  we  commented  on  one  student’s  reaction  by  posing  additional  questions  for 
students  to  explore.  [“So  what  exactly  do  you  think  is  motivating  his  actions?  And  how  can  Lawrence 
get  through  to  him  (and  everyone  else?)”]  When  instructors  participate  carefully,  through  questions 
and  comments,  students'  problem-solving  approaches  may  incorporate  more  expert-like  characteristics, 
including  the  consideration  of  the  impact  of  initial  recommendations. 

Limitations 

We  began  this  work  with  the  idea  of  identifying  patterns  in  the  development  of  students’ 
problem-solving  skills.  What  we  found,  however,  was  an  absence  of  patterns  and,  over  the  course  of 
several  semesters,  our  focus  shifted  to  examining  the  coaching  strategies  we  were  using.  In  the 
previous  section,  we  suggested  several  broad  strategies  instructors  can  use  to  guide  their  coaching.  The 
strength  of  these  suggestions,  however,  is  limited  by  several  factors,  including:  (1)  our  analysis  codes 
and  categories  may  not  have  been  sophisticated  enough  to  find  patterns  in  students’  problem  solving 
skills,  and  (2)  there  may  have  been  insufficient  time  for  students'  problem  solving  skills  to  develop. 
Both  of  these  limitations  provide  opportunities  for  future  research. 

One  of  the  unstated  goals  of  this  investigation  was  to  develop  and  apply  an  analysis 
framework  for  examining  students’  responses  to  discipline -related  case  studies.  The  framework 
developed  during  this  research,  although  greatly  refined  from  its  initial  stages,  must  still  be  considered 
preliminary.  Additional  refinements  are  needed  to  allow  consistent  application  while  analyzing 
students’  case  responses. 

The  development  of  problem-solving  expertise  is  known  to  take  place  over  a long  period  of 
time.  Trying  to  capture  this  development  in  a semester-long  course  is  difficult,  at  best.  Longitudinal 
studies  are  needed  to  determine  if,  and  how,  this  process  occurs  over  an  extended  period  of  time,  and 
the  extent  to  which  instructors  can  influence  that  process  through  the  coaching  techniques  they  use 
during  case  discussions.  Furthermore,  it  is  important  to  determine  the  extent  to  which  students  who 
develop  strong  problem-solving  skills  within  case-based  courses  actually  transfer  these  skills  to  their 
future  jobs.  Does  the  experience  of  solving  cases  in  a college  course  benefit  instructional  designers  in 
their  future  practice?  Further  research  is  needed  to  address  these  important  questions. 
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Conclusion 

Case-based  instruction  has  been  heralded  as  a powerful  means  for  helping  i nstructional  design 
students  bridge  the  gap  between  novice  and  expert  practice  (Julian  et  al.,  1999).  However,  case-based 
instruction  does  have  its  risks.  Students  may  like  using  cases  and  perceive  that  they  have  learned  a lot. 
But  this  does  not  necessarily  mean  that  they  have  gained  new  or  better  problem-solving  skills  that  will 
help  them  become  better  instructional  designers.  Blumenfeld,  Soloway,  Marx,  Krajcik,  Guzdial,  and 
Palinscar  (1991)  suggested  that  “without  adequate  attention  to  ways  of  supporting  students  and 
teachers,  leaming-by-doing  will  not  be  done”  (p.  374).  Case-based  instructors  need  effective  strategies 
for  supporting  students  during  case  study  analysis.  If  not  used  well,  cases  may  merely  perpetuate  the 
approaches  and  strategies  that  students  already  use.  Clearly,  teaching  with  cases  is  not  an  easy  task. 

As  noted  by  McNergney  (1999):  "Teaching  and  learning  with  cases  is  not  for  the  faint  of  heart"  (p.  vi). 

Sykes  and  Byrd  (1992)  stated  that  “learning  from  cases  will  depend  on  the  interaction  among 
what  the  case  presents,  what  the  reader  brings,  and  what  the  teacher  does  with  the  case”  (p.  5 1 1).  J. 
Shulman  (cited  in  Lundeberg,  Levin,  & Harrington,  1999,  p.  15)  expressed  the  importance  of 
discussion  and  facilitation  when  she  noted,  "cases,  even  with  commentaries,  do  not  teach  themselves." 
The  role  of  the  coach  (instructor)  is  viewed  as  being  critical  to  the  success  of  the  entire  process.  Yet, 
there  are  very  few  guidelines  available  for  instructors  who  wish  to  begin  using  a case-based  approach. 
With  little  guidance,  instructors  can  easily  fall  into  the  trap  of  thinking  that  just  because  case 
discussions  are  interesting  and  engaging,  that  students  are  learning  the  things  they  need  to  learn  to 
become  better  designers. 

Through  our  ongoing  examination  of  students'  responses  to  the  use  of  cases,  we  have 
identified  techniques  that  appear  effective  in  eliciting  students’  use  of  effective  problem-solving 
strategies.  Although  our  results  are  preliminary,  this  work  addresses  two  important  questions  about  the 
effective  use  of  cases  in  instructional  design  courses  including:  1)  what  to  assess  as  evidence  of 
students’  learning  in  a case-based  course  and  2)  how  to  achieve  specific  learning  outcomes  (e.g., 
increase  in  problem-solving  skills)  through  the  use  of  specific  coaching  strategies.  To  date,  little  work 
has  been  done  to  actually  measure  changes  that  occur  in  students’  problem-solving  skills  through 
participation  in  a case-based  course.  Even  less  work  has  been  done  to  tie  specific  coaching  strategies  to 
the  attainment  of  those  outcomes.  This  work  represents  the  first  step  in  addressing  these  important 
issues.  Still,  we  realize  that  there  is  much  more  to  learn.  We  plan  to  continue  to  examine  specific 
coaching  strategies  that  facilitate  student  problem-solving  within  case-based  instruction.  In  addition, 
we  plan  to  continue  to  hone  our  analysis  skills  so  that  we  can  more  readily  examine  the  effectiveness  of 
different  coaching  strategies  on  the  development  of  students'  problem-solving  skills.  Ultimately,  by 
refining  our  case-coaching  strategies,  we  hope  to  increase  the  potential  of  case-based  instruction  to 
facilitate  the  development  of  expert  problem-solving  skills  in  our  instructional  design  students. 
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Appendix  A:  Descriptions  of  Coding  Categories 

1 . Conceptualizing  the  issues  (reporting  vs.  interpreting) 

When  confronted  with  a “messy”  problem  situation,  novices  are  likely  to  report  or  summarize 
the  issues,  as  they  are  described  in  the  case.  The  report  is  often  verbatim  and  appears  to  take  the 
problem(s)  presented  in  the  case  at  face  value  with  little  apparent  effort  to  consider  the  extent  to  which 
the  presented  problems  are  at  the  “heart”  of  the  case.  The  result  is  a simple  recounting  of  issues,  as 
others  see  them,  that  often  remains  at  a superficial  level. 

Experts  are  less  likely  to  take  the  presented  problem  at  face  value.  Instead,  they  will  use  their 
own  experience  to  inform  their  view  of  the  case  and  to  reframe  the  problem,  often  using  their  own 
words.  Experts  are  more  likely  to  identify  the  “central”  issue  in  a case  or  to  categorize  or  prioritize  the 
issues  in  some  way,  often  including  an  explanation  of  the  principle  underlying  the  priorities.  The  result 
is  an  interpretation  of  the  case  that  is  often  unique  and  specific  to  the  situation. 

2.  Building  on  what  is  known  (absent  information  vs.  present  information) 

When  confronted  with  an  ill -defined  problem,  both  novices  and  experts  recognize  that  they 
have  incomplete  information.  However,  novices  tend  to  focus  on  what  they  don’t  know  and  to  ask 
questions  designed  to  fill  the  perceived  information  gaps.  In  doing  so,  they  seem  to  lose  sight  of  what 
they  do  know,  suggesting  that  a great  deal  of  additional  information  is  necessary  before  a solution  can 
be  developed.  In  addition,  their  approach  to  information  gathering  often  seems  scattered.  That  is,  they 
suggest  additional  information  without  apparent  consideration  for  the  specific  purpose  or  value  of  that 
additional  information. 

In  contrast,  experts  more  often  focus  on  what  they  do  know.  They  will  make  inferences  based 
on  available  information  and  consider  scenarios  that  are  likely  to  emerge  as  more  information  is 
obtained.  In  addition,  they  tend  to  take  a more  focused  approach  to  information  gathering.  That  is,  they 
will  look  for  specific  information  that  will  help  confirm  or  refute  their  interpretations  or  assess  the 
effectiveness  of  their  proposed  solutions. 

3.  Attending  to  the  relationships  among  factors  (laundry  list  vs.  coherent  plan) 

Both  novice  and  expert  problem  solvers  are  likely  to  consider  multiple  factors  in  a case. 

However,  novices  often  take  a “laundry  list”  approach  to  both  identifying  problems  and  proposing 
solutions.  With  problems,  this  often  takes  the  form  of  lists  of  issues  (sometimes  long  lists)  that  are 
presented  as  separate  items.  There  is  little  apparent  consideration  for  how  the  issues  might  be  related, 
perhaps  as  parts  of  a larger  issue,  and  little  apparent  effort  to  organize  the  list.  With  solutions,  this  often 
takes  the  form  of  a list  of  suggestions,  again  presented  as  separate  items  with  little  apparent 
consideration  for  the  relationships  that  might  exist  among  the  suggestions.  In  both  cases,  the  result  is  a 
piecemeal  collection  of  ideas  that  lacks  apparent  coherence  or  coordination. 

In  contrast,  experts  are  more  likely  to  have  a coherent  plan.  They  may  identify  multiple  issues, 
but  are  more  likely  to  make  explicit  links  among  those  issues.  Similarly,  they  may  suggest  multiple 
solutions,  but  link  the  suggestions  together  in  a coordinated  effort  to  solve  the  problem(s). 

4.  Considering  solutions  in  tentative  terms  (dictatorial  vs.  advisory) 

Both  novice  and  expert  problem  solvers  are  solution  oriented  and  often  propose  solutions 
early  in  the  problem-solving  process.  However,  they  will  do  this  in  different  ways.  Novices  often 
propose  solutions  in  a dictatorial  manner  (you  must  . . you  should. . .)•  They  are  likely  to  take  a 
judgmental  position,  often  criticizing  or  finding  fault  with  one  or  more  key  players  in  the  case.  In 
addition,  once  proposed,  their  suggestions  are  often  “carved  in  stone.”  As  new  information  becomes 
available,  novices  tend  to  stand  by  their  original  suggestions.  Rather  then  modifying  their  suggestions, 
they  often  find  ways  to  justify  them  in  spite  of  the  new  information. 

In  contrast,  experts  recognize  that  the  available  information  is  incomplete  and  that  there  are 
multiple  ways  to  address  the  different  issues.  As  a result,  they  are  likely  to  take  an  advisory  tone  when 
making  suggestions  (you  might...,  perhaps  you  could. They  are  more  likely  to  take  an  empathic 
position,  explaining  possible  reasons  behind  the  actions  of  the  key  players  in  the  case.  And  they  are 
more  likely  to  modify  or  eliminate  proposed  solutions  new  information  becomes  available. 

5.  Considering  the  impact/implications  of  proposed  solutions  (narrow  focus  vs.  broad 

focus) 

Both  novices  and  experts  frequently  suggest  solutions  to  the  problems  identified  in  a case. 
Novices  make  suggestions  without  apparent  regard  for  how  those  suggestions  might  be  implemented  or 
what  effects  they  might  have . That  is,  novices  typically  do  not  include  overt  consideration  of  impact 
and  implications  in  their  write-up  of  a case.  When  they  do  consider  impact/implications,  it  is  limited  to 
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a narrow  focus  on  resolving  the  identified  problem.  This  takes  the  following  form:  The  problem  is  . . . 
What  I would  do  is  . . . That  would  solve  the  problem  by  . . . 

In  contrast,  experts  typically  think  through  their  suggestions,  explicitly  considering  how  those 
suggestions  might  be  implemented  and/or  what  effects  they  might  have.  In  addition,  their  consideration 
of  impact/implications  is  broader.  It  often  goes  beyond  a straight-line  focus  on  problem  and  its 
immediate  solution  to  consider  other  issues  that  may  be  important  (e.g.,  cost,  time,  reactions  of  key 
players). 
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The  Virtual  Retina:  Is  Good  Educational  Technology  Always 

Strategic? 

Sandra  Dowie 
University  of  Alberta 


Educational  technology  units  must  continually  monitor  their  strategic  plans  to  ensure  that 
they  are  aligned  with  the  realities  of  their  institutions.  Strategic  dissonance  o ccurs  when  previously 
successful  strategies  are  no  longer  achieving  the  same  positive  outcomes.  The  Virtual  Retina  project  is 
used  here  as  an  example  of  strategic  dissonance  for  the  Academic  Technologies  Centre  (ATL)  at  the 
University  of  Alberta.  In  addition,  a number  of  methods  for  analyzing  the  strategies  used  by 
educational  technology  units  are  presented.  These  methods  provide  a means  for  units  in  higher 
education  to  conduct  the  ongoing  task  of  maintaining  their  strategic  plans. 

The  Academic  Technologies  for  Learning  Centre  (ATL)  is  the  campus  educational  technology 
unit  at  the  University  of  Alberta.  For  over  five  years,  it  has  effectively  facilitated  the  professional 
development  of  instructors  and  supported  the  production  of  educational  resources.  The  educational 
technology  environment  is  a “high  velocity”  world  with  rapidly  shifting  demands  and  expectations. 
This  necessitates  that  units  such  as  ATL  continually  monitor  the  effectiveness  of  their  strategic  plans  to 
ensure  that  they  are  aligned  with  the  realities  of  the  academic  environment.  During  this  monitoring 
process,  it  is  important  to  be  attuned  to  situations  in  which  previously  effective  strategies  aren’t 
achieving  the  expected  positive  outcomes. 

ATL’s  products  and  services,  effective  until  recently,  are  no  longer  achieving  the  same 
positive  outcomes.  This  sort  of  change  would  be  expected  in  the  dynamic  field  of  educational 
technology, 

...in  extremely  dynamic  industries  alignment  between  a firm 's  strategic  intent  and  strategic 

action  is  not  likely  to  last.  Inevitably,  strategic  actions  will  begin  to  lead  or  lag  strategic 

intent.  Such  divergences  between  intent  and  action  cause  ‘ strategic  dissonance  ’ in  the 

organization. 

(Burgelman  & Grove,  1996,  p 8) 

Strategic  dissonance  can  be  used  as  a lever  for  improvement  within  an  organization  if  it  results 
in  a thorough  analysis  of  the  organizations  external  environment  and  internal  capabilities.  This  sort  of 
analysis  is  often  referred  to  as  a strategic  analysis.  There  are  many  methods  for  conducting  a strategic 
analysis  that  will  enable  an  organization  to  improve  its  approaches  rather  than  experience  ongoing 
failure. 

The  Virtual  Retina  CD-ROM  project  is  presented  here  as  an  example  of  strategic  dissonance, 
the  Achilles  heel  of  many  educational  technology  units.  Following  this  case,  a number  of  methods  for 
analyzing  the  strategies  used  by  educational  technology  units  are  presented.  These  methods  provide  a 
means  for  units  in  higher  education  to  conduct  the  ongoing  task  of  maintaining  their  strategic  plans . 

The  Virtual  Retina:  A high-tech  case  study 

ATL  employs  twenty  staff  members  whose  range  of  expertise  includes  instructional  design, 
evaluation,  multimedia  development,  Web  authoring,  and  graphic  arts.  ATL  provides  grants  for  ATL 
partnerships  to  instructors  as  the  U of  A.  The  grant  includes  money  intended  to  support  release  time  for 
instructors  to  work  in  the  ATL  production  studio  learning  instructional  design  and  technical  production 
skills.  ATL  also  offers  a diverse  range  of  workshops  related  to  teaching  and  technology  to  the  campus. 

The  Virtual  Retina  CD-ROM  was  created  through  an  ATL  partnership  grant  provided  to  the 
Department  of  Ophthalmology  at  the  U of  A.  The  project  team  consisted  of  three  ophthalmologists,  a 
medical  student,  an  instructional  designer,  an  Authorware  developer,  and  a graphic  artist.  For  the 
Virtual  Retina  partnership,  the  grant  was  used  to  employ  the  medical  student  to  work  full  time  in  the 
ATL  production  studio  for  six  months.  The  project  lasted  from  December  1999  to  August  2000. 

The  CD-ROM  contains  a virtual  clinical  environment  to  provide  residents  with  skills  and 
experience  related  to  diagnosis  of  diabetic  retinopathy,  a leading  cause  of  blindness  in  developed 
countries.  As  some  diabetic  pathologies  are  clinically  diagnosed  by  their  3D  appearance,  problem- 
based  cases  require  learners  to  evaluate  3D  digital  photographs.  Through  this  application,  learners 
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acquire  a thorough  understanding  that  would  otherwise  take  an  extended  period  of  time  to  achieve 
through  traditional  clinical  observation.  The  Virtual  Retina  CD-ROM  contains  the  following  features: 

Case -based  problem  solving:  In  the  CD-ROM’s  Clinic,  learners  practice  diagnosing  and  prescribing 
treatments  for  a comprehensive  set  of  pathologies  without  being  dependent  on  the  availability  of  patients. 
3D  digital  imagery:  Digital  imagery  has  been  used  in  ophthalmology  for  some  time  now,  but  never  in  3D.  The 
observation  of  the  stereographic  appearance  of  several  diabetic  retinopathic  lesions  is  vital  to  diagnosing  and 
managing  patients.  A partnership  with  Stereoviewer  Inc.  has  enabled  the  incorporation  of  this  advanced 
technology. 

Interactive  tutorials:  Highly  visual  tutorials  accompanied  by  decision  trees  provide  vital  background 
knowledge  about  diabetic  retinopathy  and  illustrate  the  diagnostic  decision  making  process. 

Library  of  articles:  Learners  can  acquire  copies  of  comprehensive  literature  reviews  on  pertinent  topics  in 
diabetic  retinopathy. 

Glossary:  Key  terms  are  defined  and  hyper  linked  throughout  the  application. 

The  Virtual  Retina  CD-ROM  will  be  used  by  approximately  five  ophthalmology  residents  a 
year  at  the  U of  A.  It  will  also  be  available  at  no  cost  to  Canadian  ophthalmology  schools  of  which 
there  are  nine  English-speaking  and  two  French-speaking  programs.  The  Virtual  Retina  application  is 
sound  pedagogically  and  has  superior  production  values. 

Despite  the  obvious  merits  of  the  Virtual  Retina  CD-ROM,  the  project  has  been  the  topic  of 
much  debate  at  ATL.  One  concern  is  that  the  project  consumed  a significant  amount  of  resources,  yet 
will  only  be  used  by  a handful  of  medical  residents  a year.  How  does  an  educational  technology  unit 
evaluate  whether  to  develop  resources  that  will  be  used  by  five  ophthalmology  residents  a year  versus 
resources  that  might  support  400  first  year  students?  Does  the  Ophthalmology  department,  through  the 
sponsorship  of  charitable  foundations  such  as  Canadian  National  Institute  for  the  Blind  and  the 
Canadian  Diabetes  Foundation,  better  fund  a project  such  as  the  Virtual  Retina? 

Secondly,  while  the  ophthalmology  instructors  were  the  primary  partners  for  this  project,  they 
had  little  time  to  devote  to  contributing  content  or  being  involved  in  the  ongoing  development  effort. 
Thus,  the  U of  A faculty  members  did  not  develop  any  new  skills  and  understandingrelated  to  teaching 
and  technology,  which  is  the  intended  outcome  of  the  ATL  partnership  program.  The  medical  student 
acquired  many  skills  related  to  multimedia  development  including  multimedia  script  writing  and 
project  management.  However,  he  is  not  part  of  the  university  teaching  staff,  so  ATL’s  primary 
function  of  being  a professional  development  rather  than  production  unit  was  not  realized  with  this 
project.  This  is  an  excellent  example  of  strategic  dissonance.  ..the  intent  of  the  ATL  partnership  was 
only  partially  realized  despite  extensive  support  and  a high-quality  product  as  an  outcome.  This 
situation  was  experienced  in  other  projects  at  ATL  over  the  course  of  the  past  year. 

Are  CD-ROM  projects  inherently  too  costly  to  be  sustained  by  campus-based  technology 
centres?  Or,  should  more  of  this  high-end  type  of  production  be  supported?  How  do  organizations  such 
as  ATL  and  other  campus  educational  technology  centres  formulate  the  strategies  they  will  use  on  an 
ongoing  basis?  While  people  are  generally  aware  of  how  to  compose  mission  and  vision  statements,  the 
path  to  a clear  set  of  operational  strategies  is  one  full  of  dead-ends  and  morasses  of  detail. 

Using  strategic  analysis  strategies 

While  ATL’s  business  plan  is  full  of  noble  goals,  a precise  set  of  operational  strategies  is  lost 
within  the  exhaustive  lists  of  tasks.  Strategic  planning  involves  assessing  both  the  external  competitive 
environment  as  well  as  the  internal  capabilities  of  an  organization  to  meet  the  demands  of  the  campus 
environment.  In  the  following,  a sampling  of  methods  will  be  used  to  assess  the  internal  resources  and 
competitive  capabilities  of  ATL.  These  methods  include  strategic  mapping,  SWOT  analysis,  value 
chain  analysis,  and  competitive  strength  assessment. 

Descriptions  of  methods  for  conducting  strategic  analysis  are  usually  framed  for  business  and 
industry  environment  and  are  fraught  with  terms  such  as  “rival”,  “marketplace”,  and  “threats.”  These 
terms  are  not  commonly  used  within  academia.  However,  perhaps  they  should  be.  Often  campus  units, 
such  as  ATL,  flounder  as  they  apply  imprecise  planning  models.  A more  “business-like”  approach  to 
strategic  analysis  can  yield  many  valuable  insights  and  is  appropriate  for  the  public  sector  in  general 
and  universities  in  specific  (Poister  & Streib,  1999;  Tischler,  Biberman,  & Alkhafaji,  1 998). 

Several  questions  will  be  used  to  structure  the  discussion.  The  answers  to  these  questions 
provide  a number  of  vantage  points  from  which  to  view  ATL’s  strategies  (Thompson-Strickland, 
2000).  This  process  is  one  valuable  not  only  for  ATL,  but  also  for  other  educational  technology  units 
coping  with  the  ever  shifting  dynamics  of  educational  technology  implementation  on  their  respective 
campuses. 
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What  is  the  unit’s  external  competitive  environment? 

ATL  is  located  on  a campus  that  serves  approximately  30,000  students.  It  has  a faculty  of 
tenured  and  part-time  teaching  staff  approaching  3,000.  A number  of  other  centres  on  campus  serve 
this  population  of  instructors.  Some  provide  services  that  overlap  ATL’ s.  To  remain  viable,  ATL  must 
demonstrate  that  it  is  filling  an  important  niche  on  campus.  It  must  continue  to  differentiate  its  services 
in  a way  that  satisfies  University  Hall’s  desire  to  avoid  redundancy  and  support  only  those 
organizations  adding  true  value  to  the  university. 

Strategic  maps  are  one  way  to  visualize  the  relationship  of  an  organization  to  other 
“competitors.”  While  units  situated  on  campuses  are  sheltered  from  the  full  force  of  a free  market,  they 
are  funded  by  central  administration  and  vie  for  the  limited  funds  available.  In  this  sense,  an 
educational  technology  unit’s  competitive  market  consists  of  the  centres  that  support  teaching  and 
technology  use  on  campus.  A strategic  map  such  as  shown  in  Figure  1 depicts  how  campus-based  units 
position  themselves  related  to  supporting  academic  staff  members  at  the  U of  A. 


Figure  1 : Strategic  Map  of  a Campus  Educational  Technology  Centre 
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In  the  above  map,  technology  and  educational  support  have  been  chosen  as  the  two  axes. 
However,  any  two  variables  that  distinguish  the  activities  of  organizations  can  be  used.  A circle 
roughly  proportionate  to  the  size  of  its  operating  budget  represents  each  entity. 

The  map  demonstrates  a clustering  of  centres  in  the  lower  right  quadrant.  These  centres  are 
largely  technology  focused  with  little  emphasis  on  effective  teaching.  In  the  map,  it  is  evident  that  ATL 
serves  an  important  niche  by  providing  services  related  to  both  instruction  and  the  application  of 
technology  and  is  currently  favorably  located  on  the  map.  The  proximity  of  the  faculty-based  units  to 
ATL  illustrates  their  direct  competition  with  ATL.  A number  of  questions  that  arise  with  respect  to 
ATL  and  campus  technology  units  in  general  in  connection  to  the  strategic  map.  What  services  do  the 
units  offer  that  overlap  with  other  organizations?  How  might  ATL  operate  to  capitalize  on  its  niche  and 
maintain  or  expand  funding?  How  might  ATL  position  itselfin  relationship  to  faculty -based  units  to 
avoid  losing  funding  as  more  and  more  of  these  centres  develop  on  campus?  Some  of  the  answers  to 
these  questions  may  be  achieved  from  an  internal  analysis  of  ATL  guided  by  the  following  five  key 
questions. 

How  well  is  the  present  strategy7  working? 

The  competitive  approach  ATL  uses  is  to  provide  a differentiated  range  of  services  that  are 
not  available  elsewhere  on  campus.  Clear  standards  to  evaluate  ATL’s  success  are  absent  from  ATL’s 
strategic  plan.  However,  several  means  that  might  be  used  for  this  type  of  evaluation  are  suggested 
below. 
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Table  1 : Success  Indicators  for  ATL 


Success  Indicators 

Evaluation  of  level  of  success 

Proportion  of  target  population 
served 

On  a recent  survey,  respondents  indicated  that  22%  had  accessed  ATL’s 
services  while  40%  had  accessed  the  campus  teaching  centre’s  services. 

Cost  effectiveness  compared  to 
other  centres,  both  on  campus 
and  off 

Smaller  faculty-based  centres  may  also  be  viewed  as  more  cost-effective. 

Partners’  evaluations  of  services 

Research  is  underway  on  this  issue 

Number  of  new  media  resources 
created 

This  isn’t  tracke  d effectively.  Projects  often  take  a long  time  to  complete  and 
there  are  some,  which  do  not  result  in  the  development  of  resources. 

Utilization  of  ATL’s  production 
studio 

Below  capacity. 

Impact  on  teaching  and  learning 
on  campus 

This  is  tough  one!  But,  ultimately  an  indicator  critical  to  ATL’s  long-term 
competitive  success. 

Given  that  indicators  such  as  profit  margins  are  not  relevant  to  ATL,  one  strategy  for  this 
centre  is  to  benchmark  its  key  activities  against  other  educational  technology  centres  both  on  and  off 
campus.  To  keep  its  competitive  advantage  on  campus,  ATL  must  become  proactive  in  determining 
standards  for  measuring  its  success  and  in  developing  strategies  to  achieve  these  standards. 

What  are  the  organization’s  resource  strengths  and  weaknesses  and  its  external 
opportunities  and  threats? 

A SWOT  analysis  is  a review  of  an  organization’s  resource  strengths  and  weaknesses 
contrasted  with  its  external  opportunities  and  threats.  This  analysis  enables  planners  to  aim  to  produce 
a good  fit  between  an  organizations  capability  and  the  demands  of  its  market  situation.  ATL’s  SWOT 
analysis  appears  in  Table  2. 

Table  2:  SWOT  Analysis  for  ATL 


Potential  Resource  Strengths  & Competitive 
Capabilities 

Potential  Resource  Weaknesses  and 
Competitive  Deficiencies 

• expertise  in  instructional  development  and 
evaluation 

• technical  expertise  in  new  media 

• well-endowed  technical  facility 

• direct  contact  with  VP  Academic 

• recognized  campus  leader  in  instructional 
technology  development 

• under-utilized  facility  capacity 

• R & D not  adequately  supported  by  skilled 
staff 

• greater  emphasis  on  excellence  in  research 
rather  than  on  superior  teaching  on  campus 

Potential  Opportunities  for  ATL 

Potential  External  Threats 

• faculty-based  units  don’t  have  resources 
for  R & D 

• increasing  impetus  for  higher  education  to 
review  curriculum 

• distance  education  programs  expanding 

• ever  increasing  need  for  production 
support 

• possible  pressure  to  become  a cost-recovery 
centre 

• shift  to  early-majority  adopters  changes 
typical  client  needs 

• faculty-based  centres  might  duplicate 
services  and  request  central  funding 

• proliferation  of  workshops  related  to 
technology 

For  ATL  to  sustain  is  position  on  campus,  it  must  capitalize  on  internal  resources  that  are 
...valuable,  rare,  and  costly-to-imitate...  (Barney,  1995).  The  combined  skills  of  instructional 
development  and  educational  technology  meet  these  criteria  as  they  are  rarely  found  on  campus  and  are 
hard  to  duplicate.  These  competencies  are  further  augmented  by  ATL’s  financial  capability  to  support 
research  and  development  in  the  field  of  instructional  technology.  These  competencies  are  what  ATL 
does  best  and  will  insulate  it  from  many  of  its  external  threats.  Market  opportunities  to  heed  are  the 
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needs  of  faculty-based  units  for  support,  the  ever-increasing  impetus  on  campus  for  new  media 
production  support,  and  the  expanding  number  of  distance  education  programs. 


Are  the  educational  technology  unit’s  prices  and  costs  competitive? 

One  of  ATL’s  most  powerful  clients  is  University  Hall  as  it  funds  ATL.  Therefore,  it  is 
essential  that  ATL  demonstrate  a solid  return  on  investment  for  this  funding.  Faculty  members  are  also 
clients.  As  most  of  ATL’s  services  are  available  at  no  cost,  a critical  cost  for  faculty  members  is  time 
spent  on  projects.  ATL  must  manage  its  projects  in  a time  efficient  manner  to  satisfy  these  clients. 

A value  chain  analysis  reveals  the  core  activities  used  to  design,  produce,  market,  deliver,  and 
support  a product  or  service.  ATL’s  value  chain  is  shown  below.  The  lighter  boxes  indicate  services 
that  ATL  does  not  provide. 

Figure  2:  ATL’s  Supply  Chain 


While  value  chains  typically  show  a flow  between  one  activity  and  the  next,  an  important 
characteristic  of  ATL’s  value  chain  is  that  the  activities  are  not  necessarily  integrated  or  sequential. 
Clients  often  access  ATL  at  various  segments  of  its  value  chain.  For  example,  ATL  staff  members  are 
frequently  asked  to  provide  technical  assistance  to  faculty  members  who  haven’t  been  involved  in  any 
way  with  ATL  previously.  While  random  access  to  workshops  may  be  supported,  random  access  across 
the  value  chain  leads  to  a fragmented,  costly  approach.  Staff  work  with  bits  of  projects  out  of  context 
and  aren’t  able  to  assure  a high-quality  outcome.  As  well,  staff  members  are  continually  disrupted  in 
their  work  and  are  unable  to  complete  high  priority  projects.  ATL  has  not  been  positioned  as  a help- 
desk service  provider,  yet  much  time  is  devoted  to  this  sort  of  response  to  the  detriment  of  other 
activities.  On  a strategic  level,  ATL’s  value  chain  should  be  integrated  and  streamlined  to  avoid  falling 
into  the  something  for  everyone  trap. 

Porter  ( 1 996)  stresses  the  importance  of  making  trade-offs  and  integrating  activities  to  assure 
that  an  organization  achieves  and  maintains  its  competitive  advantage. 

There  will  be  constant  pressures  to  compromise,  relax  trade-offs,  and  emulate  rivals.  One  of 
the  leader’s  jobs  is  to  teach  others  in  the  organization  about  strategy — and  to  say  no. 


Tradeoffs  are  prerequisite  to  strategic  planning  because  of  problems  due  to: 

• inconsistencies  in  image  and  reputation 

• different  activities  require  different  product  configurations,  different  employee  behavior, 
different  skills,  and  different  management  systems 

• limits  due  to  internal  coordination  and  control 

For  ATL,  making  trade-offs  might  mean  providing  technical  support  to  only  those  clients  that 
are  involved  with  ATL  on  a complete  project.  Activities  would  also  be  aligned  to  focus  on  activities 
that  are  of  a high  value  to  ATL  strategically.  For  example,  other  campus  centres  provide  technical  help- 
desk support  or  technical  training  workshops.  ATL  should  divest  itself  of  these  activities  and  focus  on 
activities  related  to  maximizing  the  number  and  quality  of  instructional  technology  projects  it  supports. 
This  sort  of  approach  involves  a number  of  difficult  decisions  for  ATL.  Its  management  will  be  well 
advised  to  consider  Porter, 


...a  strategic  position  is  not  sustainable  unless  there  are  trade-offs  with  other  positions. 


In  all  of  this,  operational  effectiveness  must  be  assured  (Porter,  Strickland-Thompson,  2000). 
Activities  in  ATL’s  value  chain  are  often  highly  time-consuming  for  ATL  staff  members  and  their 
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( Porter,  p 77) 
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clients.  Streamlining  the  analysis,  design,  and  production  process  will  be  essentia]  to  effective 
operations.  Project  management  is  gradually  evolving  at  ATL.  It  is  a difficult  process  as  many  aspects 
of  the  university  environment  are  at  odds  with  a more  business  like  production  process. 

How  strong  is  the  centre’s  competitive  position? 

In  a business  environment,  the  success  factors  are  used  to  evaluate  a business’s  competitive 
advantage.  In  an  academic  environment,  analyzing  the  success  factors  indicates  the  strengths  and 
attributes  of  ATL  in  comparison  to  other  centres  on  campus  that  also  address  teaching  and/or 
educational  technology.  A preliminary  outline  of  factors  essential  to  ATL’s  success  is  listed  below. 

Table  3:  Key  Success  factors  for  ATL 


Factors 

Instructional  design  expertise 

Evaluation  expertise 

Technological  expertise 

Good  communications  & teaching  skills 

Reputation  on  campus 

New  product  innovation  capability 

Project  management  skills 

Academic  research  capability 

Cost  position  relative  to  other  campus  centres 

Production  resources  (hardware,  software) 

Customer  service  orientation 

These  success  factors  were  circulated  to  ATL’s  management  and  staff  who  were  asked  to  rank 
the  importance  of  each  factor.  For  several  of  the  factors,  staff  responses  diverged  significantly  from  the 
director’s.  This  suggests  that  more  discussion  needs  to  take  place  within  ATL  regarding  its  direction. 

ATL  should  undertake  to  define  its  Key  Factors  for  Success  and  reach  understanding  within 
the  organization  about  the  relative  importance  of  each  factor.  Then  ATL  can  use  this  inventory  of 
factors  to  compare  ATL’s  capabilities  to  other  campus  centres.  For  example,  if  ATL’s  production 
services  arejudged  to  bebelowpar,  while  other  centres  are  comparatively  superior,  ATL  has  at  least 
two  choices.  It  may  choose  to  outsource  the  production  aspects  of  projects  to  other  units,  or  it  could 
devote  resources  to  increasing  its  capability  in  this  area.  Choices  such  as  these  impact  whether  or  not 
ATL  undertakes  projects  such  as  the  Virtual  Retina  CD-ROM  in  the  future. 

What  strategic  issues  do  educational  technology  units  face? 

A number  of  experiences  such  as  the  Virtual  Retina  project  have  compelled  ATL  to  ask  sharp 
questions  about  the  effectiveness  of  the  various  services  it  offers.  As  a result  ATL  has  come  to  realize 
that  many  of  its  strategies  have  been  tailored  to  address  the  needs  of  educational  technology’s  “early 
adopters”  at  the  U of  A (Rogers,  1 995).  ATL’s  focused  differentiation  of  products  and  services  for  this 
group  is  increasingly  out  of  alignment  with  the  characteristics  and  needs  of  the  growing  population  of 
instructors  using  technology  who  may  be  categorized  as  being  part  of  the  “early  majority.”  The 
characteristics  of  this  model  are  summarized  below  and  contrasted  with  an  alternative  model  that  is 
more  attuned  to  the  early  majority  of  educational  technology  users  on  campus 

Table  4:  The  Shifting  Needs  of  Clients 


Client 

Early  Adopter  Model 

Early  Majority  Model 

instructor 

department 

Instructional  design 

idiosyncratic 

team  effort 

Process 

residency  at  ATL 

project  meetings 

Production 

instructor  (or  assistant) 

project  team 

Outcome 

resources  for  single  course 

resources  for  several  sections  or 
program 

Time  available 

j 

extensive 

moderate 

j Risk  at  evaluation  time  for 
! instructor 

high 

low 
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In  reconsidering  its  strategies,  ATL  must  avoid  trying  to  straddle  between  two  programs.  This 
would  occur  if  ATL  attempts  to  combine  significantly  different  approaches  to  professional 
development  with  existing  programs  and  services.  This  approach  is  doomed  to  failure.  Tradeoffs  have 
to  be  made  and  accepted  (Porter).  The  Virtual  Retina  project  demonstrated  something  that  as  occurred 
on  a number  of  ATL  projects.  The  production  model  that  required  faculty  members  to  learn  then 
produce  their  own  instructional  resources  is  no  longer  a viable  mode  for  faculty  development.  The 
early  adopters’  zeal  for  committing  to  long  term,  hands-on  production  is  not  matched  by  mainstream 
faculty  members.  Different  strategies  are  needed  for  both  production  and  faculty  development. 

In  past  strategic  planning  efforts,  it  seems  that  ATL  management  and  staff  have  been 
swamped  by  the  number  and  complexity  of  ATL’s  initiatives  and  have  difficulty  seeing  the  larger 
strategic  structure  of  ATL’s  activities.  The  analysis  methods  described  to  this  point  enable  people 
within  an  organization  to  develop  an  number  of  perspectives  on  both  their  external  competitive 
environment  and  their  internal  capabilities  to  respond  to  this  environment.  Vanguard’s  Activity  System 
Map  is  a good  next  step  for  ATL  as  it  moves  towards  refining  its  existing  strategic  plan.  This  map 
represents  the  network  of  activities  conducted  by  an  organization.  These  types  of  maps... 

. . .show  how  a company’s  strategic  position  is  contained  in  a set  of  tailored  activities 
designed  to  deliver  it.  (Porter,  p 71) 


A suggested  Activity  System  Map  for  ATL  is  shown  in  Figure  3. 


Figure  3:  ATL’s  Activity  System  Map 


provide  seek 


After  completing  the  Activity  System  Map  a technology  unit’s  management  should  ask  the 
following  sorts  of  questions: 

• Are  the  activities  consistent  with  the  unit’s  overall  positioning?  (i.e.  needs  served,  type  of 
clients  accessed) 

• How  do  the  activities  within  the  unit  reinforce  or  detract  from  each  other? 

• Could  changes  in  how  one  activity  is  performed  enable  others  to  be  phased  out? 

• What  broad  strategies  and  goals  for  the  unit  can  be  used  to  direct  these  activities? 

• What  are  indicators  for  evaluating  the  success  for  each  of  these  activities? 

With  an  Activity  Map  the  salient  features  of  an  organization  become  apparent.  Broader 
strategies  emerge  and  are  readily  translated  into  a concise  strategic  plan  for  the  organization.  At  this 
point,  educational  technology  units  are  ready  to  assess  whether  projects  such  as  the  Virtual  Retina  CD- 
ROM  are  aligned  with  the  strategic  plan. 

Conclusion 

Only  a sampling  of  possible  methods  for  analyzing  an  organization’s  external  competitive 
environment  and  internal  capabilities  have  been  explored.  There  is  a wealth  of  resources  designed  for 
business  and  industry  that  can  be  used  to  free  up  blocked  strategic  planning.  Project’s  such  as  the 
Virtual  Retina  inform  an  organization  that  it  is  entering  a period  of  strategic  dissonance,  a time  when 
successful  strategies  no  longer  work  on  all  dimensions.  At  such  times,  strategic  analysis  methods 
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facilitate  the  asking  and  answering  of  fundamental  questions  related  to  direction,  purpose,  and 
relevance  of  core  strategies  used  by  an  organization. 
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Abstract 


With  the  advent  of  the  Internet,  email  and  increased  use  of  online  services  issues  of privacy, 
anonymity  and  security  of personal  information  have  evolved  to  a new  level  of  importance. 
These  issues  have  been  discussed  extensively  as  related  to  open  access  online  services  but  are 
yet  to  be  fully  addressed for  students  of  online  classes.  This  pilot  study  sought  to  determine  if 
students  of  a web -based  distance  education  course  had  expectations  of privacy,  anonymity  or 
issues  of  sharing  personal  information  as  related  to  the  web  mediated  interactions  of  the 
online  course.  Students  taking  their  first  online  course  were  found  to  have  significantly  higher 
concerns  related  to  privacy  issues  than  those  who  had  previously  taken  online  classes.  With 
the  increase  in  web-based  distance  education  courses  and  entire  degree  programs  being 
offered  through  the  Internet  it  is  important  to  determine  student  attitudes  which  might  effect 
success  of  such  programs  and  students. 

Introduction 

With  the  advent  of  the  Internet,  email  and  increased  use  of  online  services  for  web-based 
instruction,  web  enhanced  or  supported  instruction  (such  as  chat  rooms,  streaming  video,  and  threaded 
discussions)  issues  of  privacy,  anonymity  and  security  of  personal  information  have  evolved  to  a new 
level  of  importance.  There  has  been  a great  deal  of  discussion  and  several  studies  (Bodi,  1998;  Descy, 

1 997;  Metivier-Carreiro  & LaFolIette,  1 997;  Singleton,  1 998)  conducted  concerning  privacy  and 
anonymity  of  persons  who  use  the  Internet.  For  adult  educators  Holt  (1998)  reminds  us,  these  concerns 
are  not  unique  to  the  Internet,  but  “risks  may  be  magnified  by  the  power  and  reach  of  electronic 
systems”  and  are  yet  to  be  addressed  for  students  of  online  classes. 

Distance  education  and  more  specifically  online  courses  bring  into  play  a new  set  of 
classroom  variables  and  factors  (Holt,  Kleiber,  Swenson,  Rees  and  Milton,  1 998).  Of  interest  here  are 
the  changing  attitudes  and  expectations  of  distance  education  students  who  now  are  partaking  of 
education  through  the  electronic  forum  of  the  Internet.  For  example,  within  the  traditional  classroom  a 
student’s  appearance,  and  usually  both  gender  and  race,  are  apparent  to  fellow  classmates.  The  face-to- 
face  interaction  often  found  in  traditional  educational  settings  usually  results  in  students  revealing 
components  of  their  personalities,  whether  it  is  a tendency  to  be  out  spoken  or  a lack  of  participation 
that  might  be  interpreted  as  shyness,  or  some  other  trait,  often  there  is  some  insight  into  the  person 
behind  the  student.  This  face-to-face  interaction  is  missing  in  the  online  environment  of  a web-based 
distance  education  course. 

Because  of  this  environmental  difference  between  traditional  and  online  education  we  sought 
to  determine  if  students  of  a web-based  distance  education  course  had  expectations  of  privacy, 
anonymity  or  issues  of  sharing  personal  information  as  related  to  the  web  mediated  interactions  of  the 
online  course.  For  the  purposes  of  this  study  privacy  was  defined  as  the  release  of  personal 
information  to  other  members  of  the  online  class,  including  name,  home  address,  phone  number,  and 
posting  of  their  picture  while  anonymity  was  considered  to  be  the  ability  to  conceal  one’s  identity  and 
true  personality,  even  use  of  an  alias. 

While  it  has  been  shown  that  individuals  utilizing  web-based  commerce  and  social 
interactions,  email  and  chat  rooms  have  expectations  of  privacy,  anonymity  and  personal  information 
sharing  subject  to  the  implied  security  level  of  each  activity  and  educators  note  that  such  issues  should 
be  of  concern  for  online  students  (Holt,  1 998)  the  literature  does  not  contain  specific  research  reflecting 
the  attitudes  of  students  in  web-based  distance  educational  environments  towards  these  areas  of 
personal  concern. 

Methods 
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Students  in  an  online  distance  education  course  offered  by  the  Adult  Learning  and  Technology 
Department  of  the  College  of  Education  at  the  University  of  Wyoming  were  approach  via  the 
announcements  area  of  one  of  the  course  modules  to  volunteer  to  complete  an  online  survey  regarding 
attitudes  towards  issues  of  privacy,  anonymity  and  personal  information  sharing  in  an  online 
educational  environment.  The  survey  was  given  during  the  next  to  last  week  of  the  course,  which  was 
on  distance  education  technology.  The  survey  was  built  in  Macromedia’s  Dreamweaver  3 software 
using  radio  buttons  as  the  response  method.  The  URL  for  the  survey  was  given  and  students  who 
chose  to  participate  were  asked  to  select  the  URL  and  complete  a questionnaire  consisting  of  a multiple 
choice  demographics  section  plus  1 7 attitudes  questions.  The  attitudes  questions  used  a Likert  scale  of 
5 categories  from  strongly  agree  to  strongly  disagree.  Respondents  submitted  the  completed 
questionnaires  via  the  Internet  by  choosing  a submit  button.  Likert  responses  were  weighted  1 through 
5.  Responses  to  questions  were  tallied  with  a strong  concern  for  privacy  and/or  anonymity  resulting  in 
a low  score  (1)  while  a strong  preference  for  openness  or  no  privacy  or  anonymity  resulted  in  a high 
score  (5).  Of  the  17  questions  9 were  related  to  issues  of  privacy  and  8 to  issues  of  anonymity. 
Statistics  were  performed  using  SPSS. 


Of  the  29  students  in  the  course  1 2 (4 1 .4%)  responded  to  the  online  survey,  which  is  typical  of 
online  survey  response  rates.  Of  the  respondents  58.3%  (7)  were  female  and  83,3%  (10)  were  41  years 
or  older.  A majority  of  the  students  were  married  (75%)  with  no  children  still  living  at  home  (83.3%). 
There  was  a fifty-fifty  split  between  full  time  and  part  time  student  status  and  58.3%  were  employed 
full  time.  Eight  respondents  had  never  taken  a web-based  distance  education  course  (66.7%).  Only 
16.7%  (2)  of  the  responders  had  not  taken  traditional  classroom  style  courses  at  the  post-secondary 
level.  The  majority  of  responders  were  enrolled  in  a degree-seeking  program  (66.7%)  at  the  graduate 
(50%)  or  undergraduate  level  (16.7%).  The  reasons  most  often  cited  for  taking  this  online  course  were 
the  flexibility  of  time  (16.7%),  inability  to  travel  to  the  campus  (25%),  and  that  this  course  was  only 
being  offered  as  an  online  course  at  this  time  (33.3%).  None  of  the  respondents  reported  a preference 
for  a web-based  environment  as  their  reason. 

Weighted  privacy  and  anonymity  values  were  determined  based  on  responses  and  number  of 
questions  pertaining  to  that  category  in  the  instrument.  The  scores  were  analyzed  using  independent  t- 
test.  Students  who  had  not  previously  taken  an  online  class  were  found  to  have  a significantly  lower 
mean  score  on  privacy  (showing  more  concern)  than  students  who  had  taken  2 or  more  online  classes  (t 
= 2.47,  p = . 032). 

A principal  component  analysis  extraction  of  the  data  showed  a high  degree  of  covariance, 
with  all  questions  highly  related  as  shown  by  the  consistency  of  responses.  The  strongest  reactions 
came  to  questions  such  as,  “Should  a student  be  able  to  use  a pseudonym  (as  opposed  to  their  real 
name),  for  chats  and  threaded  discussions”?  Ten  of  the  twelve  respondents  either  disagreed  or  strongly 
disagreed  (83.3%)  with  the  concept  that  students  should  be  able  to  act  anonymously.  Of  interest 
because  of  the  relative  neutrality  of  responses  are  mean  results  for  some  of  the  individual  questions  (5 
= strongly  agree,  3 = neutral,  and  1 = strongly  disagree);  security  of  information  that  is  shared  in  an 
online  class  (2.75),  live  video  and  audio  for  chats  (3.5),  sharing  biographies  at  the  beginning  of  the 
class  (3.66)  and  students  should  post  their  picture  as  part  of  the  introduction  process  (3.0).  Mild 
concern  was  expressed  over  sharing  personal  data  such  as  home  phone  numbers  or  addresses  (2.08). 

Conclusion 

While  the  sample  number  (n=12)  is  low  this  pilot  study  shows  that  students  who  had  not 
previously  taken  an  online  course  had  greater  concerns  of  privacy  than  students  who  had  prior  online 
coursework.  We  know  that  users  of  chat  rooms  and  e-commerce  expect  a certain  level  of  security 
(Moss,  2000)  it  appears  from  this  limited  study  that  perhaps  first  time  students  of  online  distance 
education  classes  have  similar  concerns  while  “experienced  online  students  do  not.  Perhaps  this  is  in 
part  due  to  the  fact  that  experienced  students  realize  they  are  entering  a somewhat  more  secure  or 
closed  environment  in  the  form  of  an  online  course. 

Refinement  of  the  questionnaire,  increase  in  the  number  surveyed  and  performing  the  survey 
again  on  both  first  time  students  and  experience  online  students  would  be  needed  to  confirm  the 
findings  of  a difference  in  attitudes  related  to  issues  of  privacy  in  these  two  groups  of  students.  With 
the  increase  in  web-based  distance  education  courses  and  entire  degree  programs  being  offered  through 
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the  Internet  it  is  important  to  determine  student  attitude  s which  might  effect  success  of  such  programs 
and  students. 
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Redesigning  The  Distance  Education  Courses 
of  Anadolu  University  in  Turkey 

Cengiz  Hakan  Aydin 
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Abstract 

Developments  in  communication  technologies  and  instructional  methods  force  distance 
education  providers  to  change  the  way  they  try  to  help  learners.  Anadolu  University  as  one  of 
the  biggest  distance  education  providers  in  Turkey  has  been  trying  to  adapt  its  distance 
education  programs  to  this  new  situation  occurred  especially  during  1990s.  The  University 
has  developed  a new  distance  education  model  and  lunched  a project  to  help  the  learners. 
This  model  includes  use  of  textbooks,  television  programs,  computer-based  instruction 
programs  on  CD-ROMs  and  Web  sites  either  together  or  alone  depends  on  the  learners  ability 
to  access  the  equipments  necessary  for  the  materials.  This  paper  aimed  to  explain  details  of 
this  ongoing  project  in  Anadolu  University. 


Introduction 


Distance  education,  the  structured  teaching-learning  process  that  is  not  limited  by  time  and 
place,  is  one  of  the  fastest  growing  applications  of  educational  technology.  This  growth  has  occurred 
especially  in  the  last  decades  of  20*  Century.  During  this  period,  developments  in  computer  based 
delivery  systems  and  in  instructional  strategies  have  caused  significant  changes  in  the  field  of  distance 
education.  Distance  learners  have  had  better  opportunities  and  freedom  to  determine  when,  where, 
what,  and  how  to  study  since  the  distance  education  providers  started  to  adapt  their  services  according 
to  these  changes. 


As  one  of  the  largest  distance  education  providers,  Anadolu  University  could  not  stay 
unaware  of  these  changes  in  the  field.  Therefore,  the  University  has  launched  a new  project  in  1999 
about  redesigning  the  distance  learning  courses  in  the  light  of  past  experiences  and  principles  of 
instructional  design. 


This  paper  aimed  to  explain  details  of  this  ongoing  project  in  Anadolu  University.  It  is  hoped 
that  the  explanations  here  will  help  readers  to  comprehend  how  a highly  content -focused  distance 
education  system  is  trying  to  transform  to  a learner-centered  system. 

First,  this  paper  will  summarize  some  of  the  trends  influencing  the  field  of  distance  education. 
Next,  it  will  give  some  of  the  current  statistics  about  the  Open  Education  Faculty  of  Anadolu 
University.  Finally,  the  paper  will  clarify  the  details  of  the  project  and  explain  what  has  been  done 
since  the  beginning  of  the  project. 


Some  of  the  Trends  in  Distance  Education 


Distance  education  has  been  influenced  by  the  developments  in  many  other  fields  since  its 
pioneers.  It  is  not  the  intent  of  this  paper  to  explain  all  these  fields  and  the  developments.  However 
there  are  two  of  them  that  have  had  major  effects  on  recent  implementations  of  distance  education. 

One  of  these  is  the  field  of  communication  technologies.  Since  distance  learners  and  the 
instructor  are  separated,  there  is  a need  for  an  artificial  communication  medium  that  will  provide  a 
channel  for  information  flow  and  interaction  between  them. 

Distance  education  providers  have  successfully  been  using  different  forms  of  communication 
technologies  for  meeting  this  need.  Print  form  was  the  first  and  the  widely  used  one,  but,  as  Spodik 
(1999)  points  out,  audio  and  video  broadcasting  technologies  helped  to  revolutionize  the  scope  and 
capability  of  distance  learning  programs.  However  all  these  have  brought  tremendous  financial  cost 
and  many  restrictions  on  the  learners. 


In  1998,  the  World  Bank  recognized  Anadolu  as  the  world’s  largest  university  (Potashnik  & Capper, 
1998). 
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On  the  other  hand,  for  many  (Harrasim,  1990;  Mclsaac  & Gunawardena,  1996;  Schrum,  1999) 
recent  developments  in  computer  based  communication  technologies  opened  new  doors  to  distance 
education  providers  due  to  their  unique  characteristics  such  as  speed  of  deliver}'  and  increased 
interaction.  The  practices  have  shown  that  computer  based  communication  technologies  helped  the 
distance  education  providers  to  reach  more  geographically  and  physically  separated  individuals,  to 
offer  more  interactivity  among  the  learners,  to  incorporate  new  instructional  methods,  and  above  all  to 
bring  the  distinctions  between  distance  education  and  traditional  education  closer  (Mclsaac  & 
Gunawardena,  1996). 

Developments  in  the  field  of  instructional  psychology  have  also  been  influencing  the  distance 
education  practices  to  a greater  extent.  Recent  studies  on  learning  have  revealed  that  learning  is  a 
change  in  meaning  constructed  from  experience  (Newby,  Stepich,  Lehman  & Russell,  1996),  and  after 
bringing  his/her  prior  knowledge  to  the  learning  experience,  the  learner  co-construct  new  meaning  onto 
former  knowledge  through  interaction  with  the  material,  with  the  other  learners  and  with  the  instructor 
(Morphew,  2000).  That  is  why  this  new  approach  has  been  called  as  constructivism,  or  constructivist 
learning.  In  general,  a constructivist  learning  activity  is  more  learner-focused  and  includes  a set  of  tasks 
and  resources  available  to  learners  for  actively  creating  their  own  meaning  in  an  authentic  context 
rather  than  passively  absorbing  knowledge  structures  created  by  the  instructor. 

Tam  (2000)  indicates  that  when  applied  to  the  distance  learning  context,  there  is  no  doubt  that 
constructivism  and  the  use  of  new  technologies  will  help  transform  significantly  the  way  distance 
education  should  be  conducted. 

"...  computer-mediated  communications,  computer-supported  collaborative  work, 
problem-based  learning  environments,  and  computer-based  cognitive  tools,  for 
example,  can  offer  the  field  of  distance  education  alternative  approaches  to  facilitate 
learning.  These  constructivist  environments  and  tools  can  replace  the  deterministic 
teacher-controlled  [content-focused]  model  of  distance  education  with  contextualized 
work  environments,  thinking  tools,  and  conversation  media  that  support  the 
knowledge  construction  process  in  different  settings.” 

Especially  during  late  1990s,  observable  consequences  and  products  of  this  approach  have 
started  to  be  seen  in  the  distance  education  programs.  For  instance,  Jonassen,  Peck  and  Wilson’s  ( 1 999) 
conceptual  model  of  learning  environments  for  technology  gives  details  and  examples  about  creating 
constructivist  distance  learning  environments.. 

Distance  Education  in  Anadolu  University 

According  to  Daniel  (1996)  Anadolu  University  is  one  of  the  mega  universities*  of  the  world 
because  of  its  nearly  600.000  enrolled  students. 

The  University  was  created  in  1981  from  an  older  institution,  the  Academy  of  Economics  and 
Commercial  Sciences  of  Eskisehir.  In  1982,  a school,  named  Open  Education  Faculty  was  founded  in 
order  to  supply  the  demand  of  higher  education. 

By  the  1982-1983  academic  year,  Open  Education  Faculty  had  enrolled  29,479  students  in  the 
fields  of  economics  and  business  administration.  By  the  1999-2000  academic  year,  the  number  of  the 
new  enrolled  students  reached  1 82,000  in  four  4 years  programs  (economics,  business  administration, 
preschool  teacher  education,  English  teacher  education)  and  1 66,000  in  22  other  2 years  programs  (e.g. 
export,  banking  and  insurance,  public  relations,  book  keeping,  tourism,  hotel  management,  sale 
management),  two  if  which  are  designed  for  the  Turkish  citizens  living  in  different  cities  of  Europe. 

The  Open  Education  Faculty  is  legally  required  to  accept  any  applicant  who  scores  at  least 
105  on  national  entrance  exam  (average  score  is  around  160).  There  is  an  idea  about  taking  anyone 
who  wants  to  get  in  any  program  of  the  Faculty,  but  has  not  been  applied  yet. 

The  Open  Education  Faculty  uses  mostly  printed  texts  and  television  programs  as  the  course 
materials.  All  these  materials  are  produced  in-house.  The  Faculty  has  printing  facilities  and 
professional  TV  studios  for  the  production  of  distance  learning  materials.  More  than  400  textbooks 
have  been  printed  and  2.200  TV  programs  have  been  produced  since  the  foundation.  The  Faculty 
broadcast  nationwide  six  hours  of  programming  everyday  on  Channel  4 of  the  Turkish  Radio  and 
Television  Corporation. 


* Mega-universities  are  large  open  universities,  each  of  which  enrolls  more  than  100,000  students  per 
year. 
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In  addition,  there  are  81  administrative  centers  throughout  the  country  and  in  58  of  them 
students  can  attend  noncompulsory  evening  classes  several  times  per  week  to  get  the  academic 
counseling. 

The  Faculty  tries  to  bring  new  technologies  into  the  system.  For  instance,  a pilot  marketing 
course  was  delivered  through  videoconferencing  at  a school  in  Kazakhstan.  Also,  in  computer  based 
instruction  department,  several  samples  for  different  courses  are  produced  each  year.  However  the 
system  heavily  relies  on  textbooks. 

One  of  the  main  problems  cf  the  Open  Education  Faculty  is  the  low  graduation  rate. 
According  to  most  recent  statistics,  only  34%  of  those  who  enter  the  two  years  program  earn  degrees 
within  two  years,  and  just  23%  of  those  who  begin  four-years  programs  in  four  years. 

Studies  have  shown  that  76%  of  the  students  holds  part  and  full  time  jobs  and  having 
difficulty  for  leaving  time  for  study.  It  is  also  determined  that  most  of  the  students  are  not  using  the 
textbooks  designed  and  written  by  the  Faculty  of  Open  Education.  Instead  the  students  find  the  books 
that  are  published  by  the  private  companies  easer  to  read  and  understand,  and  also  more  helpful  for 
themselves.  It  has  been  noticed  that  the  main  goal  of  these  books  is  to  prepare  students  to  the  exams 
instead  of  instruct.  Thus,  they  include  few  explanations  about  the  content  but  lots  of  practice. 

The  students  list  the  problems  about  the  Faculty  of  Open  Education’s  textbooks  as  being  very 
complicated,  including  long  texts,  having  academic  language,  requiring  lots  of  time  to  study,  and 
providing  few  practice.  The  students  indicated  that  because  of  these  reasons  they  think  the  books  are 
hard  to  read  and  understand. 

The  Project 


The  low  graduation  rates,  critics  about  the  delivery  systems,  and  trends  in  distance  education 
have  forced  the  Open  Education  Faculty  to  redesign  its  courses.  Therefore,  the  Faculty  has  launched  a 
new  project  in  1999  about  redesigning  the  distance  learning  courses  in  the  light  of  past  experiences  and 
principles  of  instructional  design. 

The  project  consists  of  four  stages:  Firstly,  most  of  the  Open  Education  Faculty  programs 
started  in  different  years  so  that  each  curriculum  developed  separately.  As  a result  of  this  many  courses 
that  included  almost  the  same  content  were  offered  and  materials  for  these  courses  were  produced.  This 
was  nothing  but  only  vast  of  time,  money  and  effort.  Thus,  in  the  first  stage  of  the  project,  it  was 
planned  to  re-develop  the  curriculum  of  each  program  and  determined  to  give  the  priority  to  the  four 
years  programs  (especially  economics  and  business  management).  These  courses  re-developed  and 
standardization  secured  among  the  programs.  During  this  stage,  content  specialists  worked  with 
instructional  designers  to  determine  the  main  goal  and  objectives  of  each  courses  in  the  programs. 

Secondly,  as  a result  of  critics  about  the  current  materials,  the  administration  of  the  Open 
Education  Faculty  wanted  to  provide  more  appealing,  effective,  and  efficient  materials  to  the  students. 
After  several  analyses,  it  has  been  decided  to  use  multiple  media  formats  for  each  course  because  of  a 
constructivist  idea  that  advices  proving  multiple  representations  of  the  knowledge  to  the  learners  for 
helping  them  in  constructing  their  knowledge.  A textbook  with  an  accompanying  computer-based 
instruction  program  on  CD-ROM,  a television  program,  and  a web  site  were  the  media  selected  for  the 
courses. 

According  to  this  selection,  each  medium  is  responsible  for  the  different  aspects  of  instruction 
process.  In  other  words,  textbook  is  the  main  source  for  providing  information  to  the  learners  while 
television  program  shows  the  real  life  examples  related  to  the  information  supplied  through  the 
textbook.  Computer-based  instruction  program  on  CD-ROM  is  designed  to  overcome  the  lack  of 
interactivity  between  the  learner  and  the  material.  The  Web  site,  on  the  other  hand,  that  allow  both 
synchronous  and  asynchronous  communications  not  only  among  learners  but  also  between  learners  and 
instructor  is  intended  for  the  solution  of  other  interactivity  problems  (learner  to  learner  and  learner  to 
instructor).  The  main  idea  behind  these  sorts  of  uses  of  media  is  to  help  the  learners  to  get  advantages 
of  different  media  attributes  (Kozma,  1994). 

However,  it  has  been  found  out  that  most  of  the  distance  learners  in  Turkey  do  not  have  an 
opportunity  to  access  a computer  and/or  Internet.  According  to  recent  figures  only  8%  of  the  Open 
Education  Faculty  students  have  access  to  computers  although  the  Faculty  has  established  1 4 computer 
facility  centers  throughout  the  country  since  the  early  1990s. 

Also,  some  of  the  learners,  especially  who  live  in  rural  areas  of  Turkey,  are  not  able  to  watch 
the  television  programs  for  varying  reasons. 
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In  order  to  overcome  these  problems,  the  all  materials  (textbooks,  computer-based  instruction 
programs,  television  programs  and  Web  sites)  are  designed  in  a manner  that  each  of  them  can  provide 
a!!  aspects  of  instruction  process  (e.g.  motivating  learners,  providing  information,  giving  practice  and 
feedback,  etc.).  For  instance,  Web  site  has  several  links  that  takes  learners  to  sites  where  they  can  get 
information  about  the  topic,  have  opportunity  to  practice  and  get  immediate  and/or  delayed  feedback, 
and  so  on. 

After  selecting  and  designing  the  instructional  media,  thirdly,  a pilot  project  is  planned  to  be 
done  in  order  to  see  effectiveness  and  appeal  of  materials  produced  according  to  instructional  design 
principles.  An  instructional  design  team  including  an  instructional  designer,  a graphic  artist  and  a 
content  specialist  was  formed  in  summer  1999.  The  main  duties  of  this  team  were  determined  as 
preparing  a model  chapter  and  conducting  a pilot  test  to  see  if  the  new  approach  (designing 
instructional  materials  according  to  the  instructional  design  principles  and  theories)  has  positive  effects 
on  Open  Education  students. 

The  team,  first,  selected  “law  of  demand  and  supply”  as  the  topic  of  model  chapter  because 
the  students  studying  in  different  Open  Education  programs  found  it  as  one  of  the  most  important  and 
difficult  topics.  Then,  the  team  prepared  a model  textbook  chapter  according  to  instructional  design, 
visual  communication,  distance  learning  and  adult  learning  principles.  As  a result  of  this,  many  new 
features  such  as  advance  organizers,  attention-focusing  devices,  follow  up  activities,  embedded  tests, 
new  graphical  look  are  used  in  the  chapter.  Then  a TV  program  specialist  and  a computer  based 
instruction  expert  joined  the  team.  They  also  worked  collaboratively  with  the  other  members  of  the 
team. 

After  preparing  all  the  materials,  the  team  called  the  Open  Education  students  in  Eskisehir  (a 
town  where  the  University  is  located)  to  participate  the  pilot  test.  During  the  test,  15  voluntary 
participants  are  asked  to  study  the  chapter  at  home  or  work  and  take  notes  about  the  strengths, 
limitations,  and  time  spend  to  study.  Unfortunately  the  participants  could  not  reach  the  tv  program  and 
the  web-based  instruction  materials  due  to  technical  problems  and  the  traditional  belief  of  seeing  the 
textbook  main  means  of  instruction.  However  they  had  chance  to  review  the  printed  material.  After 
completing,  the  students  are  asked  join  a face-to-face  conversation  about  the  materials  and  asked  to 
mention  again  their  ideas  about  the  model  chapter.  Also,  a test  consist  of  items  related  to  the  topic 
covered  in  the  chapter  administered  to  the  participants  to  see  how  well  the  model  helped  them  to  learn 
the  content. 

The  participants’  high  scores  and  their  positive  remarks  on  the  model  chapter  satisfied  the 
team  and  the  administration  of  the  Faculty.  All  the  participants  got  very  high  scores  at  the  test 
administered  right  after  they  completed  studying  the  materials.  Some  of  the  participants  took  the 
scheduled  mid-term  exams  after  the  study  and  they  verbally  expressed  that  they  did  not  have  any 
problem  answering  the  questions  about  the  “Law  of  Demand  and  Supply”. 

All  the  participants  also  liked  the  idea  of  using  an  advance  organizer,  which  was  a short 
beginning  of  an  article  taken  from  a daily  newspaper.  The  participants  also  found  the  recaps  and  the 
embedded  test  very  beneficial;  follow  up  activities  and  web  addresses  very  useful.  On  the  other  hand, 
they  thought  that  the  warm-up  questions  at  the  beginning  were  quite  difficult  and  were  not  interesting. 

In  addition,  during  the  personal  conversations  the  participants  mentioned  that  the  well- 
designed  tv  and  computer  based  instruction  programs  might  be  very  useful  for  the  ones  who  have 
access  to  these  programs. 

Later,  fourthly,  the  Faculty  administration  decided  to  use  new  materials  during  the  2001-2002 
academic  year.  So  that,  course  materials  design  teams  were  formed  for  each  course  and  they  started  to 
work  on  the  course  materials  on  the  bases  of  the  model  chapter  on  ’’Law  of  Demand  and  Supply”, 
which  was  revised  in  the  light  of  the  participants  suggestions,  under  the  supervision  and  coordination 
of  the  instructional  design  department. 

The  core  team  usually  consists  of  an  instructional  designer,  a content  specialist,  a graphic 
artist  and  a coordinator.  Besides  tv  program  specialists  and  web/computer  based  instruction  experts  are 
joining  the  team. 

According  to  the  plans,  these  teams  firstly  focus  on  the  textbook  because  it  is  a fact  that 
textbooks  are  the  ones  almost  all  students  easily  reach.  They  start  training  writers  about  the  new 
structure  of  the  course  materials  and  instructional  design  principles  (advance  organizers,  instructional 
objectives,  etc).  After  receiving  the  written  chapter,  the  team  members  meet  with  graphic  artists,  and  tv 
and  web/computer  based  instruction  specialists.  While  graphic  artists  add  visuals  into  the  text  the  tv 
program  s,  web  pages  and  computer  based  instruction  specialist  starts  producing  the  programs.  The  core 
team  members  secure  the  interrelations  among  all  these  materials.  After  completing  all  the  materials, 
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they  are  revised  by  the  course  team.  The  team  offer  revisions  if  there  is  a need.  After  revisions  and 
duplicating,  all  the  materials  are  packed  together  as  the  course  materials. 

But,  due  to  lack  of  enough  time  and  qualified  human  resources,  production  of  web/computer 
based  instruction  programs  were  postponed.  So  the  project  has  continued  with  the  production  of 
textbooks  and  tv  programs. 

There  have  been  some  other  obstacles  in  the  design  and  development  procedure,  too.  One  of 
the  very  important  one  is  the  limitation  of  the  number  of  the  instructional  designers  in  Turkey, 
especially  in  Anadolu  University.  This  number  is  limited  only  to  the  fingers  of  a hand.  Due  to  this 
limitation,  each  instructional  designer  had  to  be  in  different  course  teams.  This  caused  the  problem  of 
not  focusing  on  the  each  course  intensively.  However,  during  the  project  several  new  graduates  of  the 
Educational  Communications  and  Planning  Department  o Anadolu  University  helped  these 
instructional  designers  and  through  this  participation  they  got  training  about  the  instructional  design 
and  instructional  media.  For  the  other  courses  they  will  take  a part  in  course  teams  as  the  instructional 
designers. 

Another  important  obstacle  was  the  other  ongoing  projects  in  the  University  and  the  Faculty. 
Because  of  those,  several  facilities  such  as  tv  studios  and  printing  office  could  not  be  used  by  the  team, 
and  some  of  the  project  members  such  as  graphic  artists  and  content  specialists  could  not  join  the 
whole  procedure.  As  a consequence  of  this,  all  the  timing  plans  failed  and  left  a few  times  -like  3 
months-  to  design  and  development  of  the  courses.  So  that,  the  Faculty  administration  has  decided 
apply  this  project  on  mathematics,  economics  and  business  management  courses.  Now  course  teams 
are  working  on  instructional  materials  of  these  courses. 

Conclusion 

Developments  in  communication  technologies  and  instructional  methods  force  distance 
education  providers  to  change  the  way  they  try  to  help  learners.  Anadolu  University  as  one  of  the 
biggest  distance  education  providers  in  Turkey  has  been  trying  to  adapt  its  distance  education  programs 
to  this  new  situation  occurred  especially  during  1990s.  The  University  has  developed  a new  distance 
education  model  and  lunched  a project  to  help  the  learners.  This  model  includes  use  of  textbooks, 
television  programs,  computer-based  instruction  programs  on  CD-ROMs  and  Web  sites  either  together 
or  alone  depends  on  the  learners  ability  to  access  the  equipments  necessary  for  the  materials. 

Although  everything  planned  at  the  beginning  of  the  project  could  not  be  done  due  to  several 
reasons,  it  can  be  easily  asserted  that  this  effort  has  several  important  effects  on  Turkish  distance 
education  system. 

First  of  all,  it  introduced  the  field  and  the  process  of  instructional  design  to  the  people  who  are 
developing  products  (textbooks,  web  based  education,  computer  based  education,  educational  tv 
programs)  for  instructional  purposes  and  unaware  of  instructional  design.  Also,  it  has  revealed  the 
strength  of  instructional  design  to  the  people  who  are  aware  of  instructional  design  but  do  not 
appreciate  it.  Besides  it  gave  a clear  idea  about  what  kinds  of  features  a textbook  should  have  and  what 
sorts  of  processes  should  be  completed  during  the  development  of  an  effective  textbook.  These  are 
significant  for  the  development  of  the  instructional  design  field  in  Turkey.  Now  more  and  more  people 
have  started  talking  about  systematic  and  systemic  design  of  instruction. 

Second,  after  this  effort,  both  Faculty  of  Open  Education  and  the  University  have  appreciated 
the  dynamic  nature  of  distance  education  field  and  decided  to  keep  up  developments  in  the  field.  That 
means  the  Faculty  is  trying  to  improve  its  services  through  the  effective  use  of  instructional  media  and 
giving  up  not  to  looking  at  the  reasons  of  low  graduation  rates.  Thus,  a new  department,  instructional 
design  department,  established  under  the  Open  Education  Faculty  and  new  instructional  designers  are 
employed. 

However,  all  these  are  not  enough.  In  order  to  remove  all  kinds  of  barriers  on  the  applications 
of  more  effective,  efficient , and  appealing  delivery  methods  and  technologies,  the  administrators  of  the 
distance  education  providers  must  take  serious  steps. 
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Accessing  CBT  Learners'  Study  Skills 


Leaunda  S.  Hemphill 
Prairie  State  College 


Purpose 

The  purpose  of  the  study  (Hemphill,  2000)  was  to  determine  if  learners  who  were  given 
individualized  learner  strategies  based  on  a survey  of  their  needs  at  the  time  ofinstruction  score  higher 
than  those  learners  who  were  not  given  the  strategies.  A proof-of-concept  learner  profile  instrument 
was  developed  to  provide  specific  strategies  to  learners  to  help  them  increase  their  effectiveness  in 
taking  computer-based  training  (CBT)  courses.  The  short  survey  assesses  the  learners'  learning  needs 
and  generates  a customized  learner  profile  with  individualized  strategies  for  each  learner's  training 
needs,  learning  and  environmental  preferences,  and  optimum  use  of  learner-controlled  program 
features  for  the  task  at  hand. 

Rationale 


In  learner-controlled  environments  such  as  computer-based  training,  learner  involvement  is 
important  to  make  the  training  meaningful.  Unfortunately  many  learners  are  not  capable  of  selecting 
appropriate  learning  strategies  and  software  options  that  enhance  their  instruction  (Clark,  1984;  Yoon, 
1 994).  Individual  differences  such  as  prior  knowledge  of  related  tasks  and  motivation  for  completing 
the  task  appear  to  be  good  predictors  of  future  achievement  (Jonassen  and  Grabowski,  1 993;  Garcia 
and  Pintrich,  1 995).  Metacognitive  strategies  also  allow  learners  to  perform  effectively  in  self- 
regulated  environments  (Gay,  Trumball,  & Mazur,  1991).  Self-efficacy,  cognitive  strategies,  and 
learning  styles  may  also  affect  learning.  A number  of  researchers  (Hannafin  & Sullivan,  1 996; 
Nathenson  & Henderson,  1980)  have  recognized  the  need  for  a preinstructional  survey  to  assess  the 
learners'  individual  characteristics  and  needs  at  the  time  ofinstruction  and  then  provide  the  learners 
with  a profile  of  individualized  strategies  for  improving  their  achievement.  Unfortunately,  available 
learner  surveys  usually  focus  on  the  general  prescriptive  and  evaluative  purposes  and  are  not 
appropriate  for  suggesting  learning  strategies  for  specific  CBT  training  applications. 

Procedure 

The  learner  profile  and  survey  instrument  prototype  was  developed  and  then  tested  at  three 
different  colleges  to  determine  if  there  was  a relationship  between  the  use  of  the  suggested  learner 
strategies  and  the  learners'  achievement  on  the  posttest.  All  subjects  took  the  learner  profile  survey,  a 
pretest,  the  software-training  course,  a posttest,  and  an  exit  survey.  The  treatment  groups  also  received 
learner  profiles  with  individualized  suggested  strategies.  The  subjects  in  the  treatment  groups  were 
rated  on  their  compliance  to  the  suggested  strategies. 

Data  Analysis 

The  experimental  analysis  was  as  follows.  Pretest  and  posttest  scores  were  compared  using  a 
t-test  and  descriptive  statistic  s,  as  was  the  length  of  time  the  subjects  spent  completing  the  training.  The 
treatment  group  subjects'  and  raters’ judgment  of  the  subjects'  compliance  to  following  the  strategies 
suggested  were  correlated.  Descriptive  statistics  were  done  on  the  treatment  group  subjects'  opinion  of 
which  strategies  were  the  most  useful  and  least  useful.  The  instrument  questions  analysis  was  as 
follows.  The  distractability  and  perceptual  modality  construct  survey  items  were  analyzed  using 
frequency  data.  A principal  component  analysis  of  the  results  of  the  pilot  group  was  used  to  analyze 
the  construct  validity  of  the  items  related  to  the  motivation,  self-efficacy,  and  metacognitive  awareness 
constructs.  Those  items  that  fit  together  in  a common  construct  were  used  to  determine  which 
strategies  the  learners  received  in  their  learner  profile. 
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Results  and  Discussion 


The  findings  of  the  experimental  study  revealed  that  there  was  a statistically  Significance 
difference  between  the  achievement  score  percent  of  subjects  who  followed  the  learning  strategies 
suggested  in  their  learner  profiles  and  of  subjects  who  were  not  given  the  strategies  or  did  not  use  the 
strategies.  The  time  spent  in  the  course  and  the  students’  rating  of  the  usefulness  of  the  training  course 
did  not  vary  much  across  the  groups. 

Interestingly,  the  importance  of  learners  complying  with  the  strategies  given  and  how  to 
measure  that  compliance  emerged  as  an  important  component  of  this  study.  The  reasons  for  not 
following  the  instructions  included  the  learners  believing  that  the  strategies  were  not  useful  to  them, 
that  working  through  a strategy  took  too  long,  and  that  they  already  had  strategies  they  used  and  they 
did  not  want  to  try  others.  This  study  may  encourage  educators  and  developers  to  look  more  closely  at 
suggesting  strategies  that  are  unique  to  each  learner  at  the  time  of  instruction  and  at  the  importance  of 
having  the  learners  use  the  suggested  strategies. 
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Online  Staff  Development:  a Collaborative  Approach 


Dusti  Howell 
Bill  Stinson 

Emporia  State  University 


Introduction 

The  public  has  raised  numerous  concerns  regarding  the  quality  of  teaching  in  our  schools. 

Political  and  professional  factions  have  developed  outcomes  and  standards  for  assessing  not  only 
student  learning  but  also  teaching  performance.  To  enhance  our  K-12  educational  systems,  a variety  of 
professional  approaches  are  being  explored  and  proposed.  For  example,  the  National  Commission  on 
Teaching  & America’s  Future’s  1 996  report,  uWhat  Matters  Most:  Teaching  and  America’s  Future,” 
outlines  several  changes  in  professional  development  programs.  One  change  proposed  is  to  organize 
teacher  education  and  professional  development  around  standards  for  students  and  teachers.  Another 
suggestion  is  developing  teacher  academies,  partnerships  of  schools/universities,  and  learning  networks 
transcending  school  boundaries.  Another  example  is  the  California  Formative  Assessment  and  Support 
System  for  Teachers  (CFASST)  which  emphasizes  the  point  of  investing  in  teacher  development  and 
sustained  examination  of  teaching  performance  over  time  will  increase  success  for  students  and 
teachers  alike  (Oliebe,  1999). 

According  to  the  NASSP  Bulletin,  October  2000,  “An  Ambitious  Vision  of  Professional 
Development  for  Teachers,”  quality  teacher  professional  development  has  never  been  more  important 
with  the  ever,  increasing  intensity  of  challenges  and  expectations  for  quality  education.  (Ganser,  2000). 
Darling-Hammond  and  McLaughlin  (1996)  expressed  increasing  teacher  effectiveness  and  student 
achievement  will  require  teachers  to  learn  not  only  new  skills  and  perspectives  but  also  unlearn 
practices  and  beliefs  that  they  have  held  onto  for  years.  Guskey  and  Sparks  (1996)  use  the  terms  - 
content,  process,  and  context  - as  guidelines  in  developing  quality  professional  development.  Content 
in  a professional  development  offering  should  deepen  one’s  understanding  of  an  academic  discipline 
and  teaching/leaming  principles.  How  these  professional  development  experiences  are  planned, 
organized,  and  implemented  reflects  the  process.  Corcoran  (1995)  describes  the  most  effective 
experiences  in  professional  development  occur  when  teachers  are  engaged  in  designed,  intensive 
intellectual,  social,  and  emotional  experiences.  The  third  quality,  context,  is  aimed  at  providing  an 
inservice  environment  for  improving  teaching  through  such  things  as  accessibility  to  resources, 
meeting  individual  and  organizational  needs,  and  learning  activities  that  are  experiential  in  nature. 

Professional  development  should  reflect  societal  demands  and  skills  needed  for  successful 
participation  in  one’s  culture  for  teachers  and  students  alike.  Technology  has  become  the  driver  in 
much  of  our  cultural  environment.  Teachers  must  not  only  understand  the  implications  of  our  existing 
technological  society  but  also  use  its  tools  in  a variety  of  ways  to  show  their  usefulness  in  teaching  and 
learning.  Schofield  ( 1 995)  describes  a s ituation  all  too  familiar  that  computers  do  not  fulfill  our 
expectations  in  schools  because  teachers  are  not  shown  how  to  integrate  this  new  technology  into  their 
instruction,  or  into  their  students’  learning  processes.  In  describing  a “generational  model  for 
professional  development,”  Caverly,  Peterson,  and  Mandeville  (1997)  cite  we  must  not  just  “train” 
teachers  in  technology  but  “educate”  then  to  think  differently  about  how  to  use  it  in  their  classrooms. 
In  a special  eSchool  News  report,  “Professional  Development:  Bringing  Technology  into  the 
Classroom  Effectively,”  it  states  that  teachers  use  computers  regularly  at  home  and  at  school  for  lesson 
planning,  correspondence,  and  personal  business.  However,  the  vast  majority  of  them  are  not  using 
them  in  their  classrooms  for  instructional  purposes  (Levin-Epstein,  2000). 

We  need  to  insure  that  teachers  are  capitalizing  upon  the  recent  technological  advances  and 
tools  for  learning.  When  teachers  are  actively  involved  in  using  and  applying  technological  tools  in 
various  learning  situations,  the  possibilities  of  using  technology  in  one’s  own  classroom  is  enhanced. 
The  following  case  study  describes  a collaborative  attempt  to  incorporate  the  use  of  various 
technological  tools  in  delivering  an  onsite  version  of  a traditional  professional  development  program. 
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A Case  Study  of  Virtual  Staff  Development 


The  Flint  Hills  Special  Educational  Cooperative  program  director  contacted  teacher  educators 
in  the  Teachers  College  at  Emporia  State  University  about  the  possibility  of  faculty  being  involved  in  a 
program  called  Fundamental  4MAT  Level  One  training.  This  training  was  made  possible  through  a 
grant  addressing  the  integration  of  learning  styles  and  the  latest  information  on  brain  research  into 
lesson  planning.  During  this  inservice  training,  corporate  trainers  from  About  Learning  Inc.,  expressed 
an  interest  in  making  this  training  accessible  to  individual  teachers  in  rural  and  international  areas. 
Subsequent  discussions  involving  Emporia  State  University  professors,  the  Flint  Hills  Special 
Educational  Cooperative,  About  Learning  and  MacLean  Media  led  to  a collaborative  distance  learning 
venture. 

Each  organization  brought  special  ingredients  to  the  potential  development  of  the  online 
delivery  of  Fundamental  4MAT  Level  One  training.  Emporia  State  University  is  the  premier 
preservice/inservice  teacher  training  institution  in  the  State  of  Kansas.  It  is  always  looking  for  more 
effective  ways  to  address  needs  and  interests  of  teachers  in  the  state.  In  addition,  increasing  interest  in 
technology  integration  within  the  K- 12  classroom  setting  and  exploring  the  potential  of  a distance 
learning  delivery  model  for  teacher  training  are  topics  of  continuing  interest.  The  Flint  Hills  Special 
Educational  Cooperative  provides  resources  and  in-services  for  eight  local  school  districts.  The 
Cooperative  is  always  looking  for  a more  effective  way  to  serve  a broad  geographical  area  of  clientele 
and  researching  ways  to  make  staff  development  more  economically  and  educationally  effective. 
About  Learning,  Inc.  is  an  established  private  corporation  which  developed  a product  called 
Fundamental  4MAT  training.  This  training  involves  learning  styles,  diversity  in  the  classroom  and 
lesson  plan  design.  About  Learning,  Inc.  expressed  an  interest  in  economically  delivering  it’s 
fundamental  training  to  teachers  in  remote  areas  via  distance  learning.  MacLean  Media  is  a broad 
based  company  involved  in  producing  a variety  of  educational  materials.  Among  its  clients  is 
Aboutleaming,  Inc. 

After  much  discussion  between  the  collaborative  partners,  a decision  was  made  to  pursue  the 
possibility  of  converting  a traditional  face-to-face  training  program  into  an  online  experience.  This 
staff  development  endeavor  would  utilize  university  faculty,  school  district  personnel  and  corporate 
partners.  To  convert  a traditional  training  program  into  an  online  experience  created  several  concerns. 
The  first  concern  was  focusing  on  adapting  written  and  audiovisual  materials  developed  by 
Aboutleaming,  Inc.  into  web  course  delivery  system  with  CD-ROM  supplementation.  Emporia  State 
University  teacher  educators  investigated  the  four  learning  styles  that  are  assessed  within  the  4MAT 
system,  mapping  out  the  various  technologies  that  would  be  most  appropriate  for  each  style.  Type  One 
learners  are  the  most  social  of  the  four  types.  They  are  interested  in  real  time  collaboration  and 
communication  within  a classroom  setting.  It  was  felt  that  live  chat,  threaded  discussion  forums,  video 
conferences,  and  e-mail  would  be  tools  that  would  allow  Type  One  learners  to  operate  within  the 
comfort  zone  of  their  learning  style.  Type  Two  learners  are  more  interested  in  the  facts  and 
information  concerning  the  course.  It  was  reasoned  that  online  surveys,  streamed  lectures,  formal 
discussions,  and  a very  structured  technology  framework  would  benefit  them  the  most.  Type  Three 
learners  prefer  to  “just  do  it.”  They  prefer  to  be  active  participants  in  a very  structured  learning 
process.  Utilizing  step-by-step  tutorials,  working  with  relevant  software  tools,  and  surfing  the  Internet 
in  a structured  way  would  be  activities  that  would  match  best  with  their  learning  style.  Type  Fours  are 
those  who  are  constantly  coming  up  with  new  ideas.  They  hate  to  be  limited  by  a structured  approach, 
preferring  to  allow  their  creative  energies  to  flourish  in  anyway  they  like.  Unstructured  chat,  open 
forum  discussions,  paint  programs,  and  web  page  development  outside  of  the  typical  template  approach 
are  activities  that  lend  themselves  to  this  type  of  learner. 

Because  of  the  interactive  nature  of  the  traditional  training  approach,  it  was  imperative  to 
provide  cooperative  learning  and  student  interactive  experiences,  to  preserve  the  integrity  of 
Aboutleaming’s  program. 

We  chose  to  focus  on  several  interactive  vehicles  in  the  development  of  this  online  course:  e - 
mail,  threaded  forum  discussions,  and  live  chat.  Live  chat  can  be  a difficult  medium  to  utilize 
successfully,  and  at  best  is  often  labeled  as  controlled  chaos.  We  chose  to  use  live  chat  for  topical 
discussions,  mixed  and  homogenous  learning  style  groups,  and  special  invited  speakers.  It  was  made 
clear  to  the  students  that  the  chat  rooms  were  also  available  to  them  whenever  they  wanted  to  work 
with  their  partners  discussing  their  lesson  plan  projects. 

One  major  component  of  the  training  required  lesson  plan  assistance  and  evaluation 
opportunities.  This  is  critical  as  students  both  develop  and  critique  each  other’s  final  projects — a 
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4MAT  lesson  plan  built  around  the  four  learning  styles,  including  left  and  right  brain  activities. 
Students  participated  in  a class  brainstorming  forum  to  create  their  initial  ideas.  They  exchanged  lesson 
plan  ideas  and  critiques  via  e-mail  with  their  partners.  To  facilitate  this  process,  a separate  website 
(http: //wwwr .lessonbank.com/)  was  developed  to  allow  students  a structured  area  to  post  and  evaluate 
lesson  plans.  At  this  site  their  final  postings  are  open  for  class  review.  Interestingly,  this  site  is  now  a 
repository  of  hundreds  of  4MAT  lesson  plans  from  both  students  and  trainers  available  to  anyone 
interested  in  lesson  plan  design. 

What  were  the  impressions  of  teachers  in  our  case  study  toward  online  delivery  professional 
delivery  courses?  Teachers  liked  the  highly  organized,  logical  structure  of  the  online  product.  Having 
an  overall  course  outline  on  the  course  website  enabled  teachers  to  plan  ahead  and  have  the  option  of 
anytime,  anywhere  to  fit  their  daily  schedules  to  complete  assignments.  Teachers  favored  the  multi- 
dimensional resource  applications  of  using  traditional  hard  copy  materials,  links  to  internet  information 
and  a comprehensive  CD  containing  self  evaluative  assessment  instruments,  graphical  representation 
and  audio-visual  mini-lectures  of  related  materials.  Online  forum  discussions  were  looked  upon 
favorably  as  opportunities  to  read  and  digest  information  before  having  to  respond  to  classmates.  Also, 
it  was  indicated  that  this  form  of  discussion  could  not  be  dominated  by  a very  few  or  curtailed  by  a 
busy  onsite  class  schedule.  It  allowed  everyone  input  on  significant  course  issues  - a true  expression 
of  diversity.  Finally,  many  course  participants  expressed  that  online,  distance  learning  was  the  wave  of 
the  future  in  education. 

Besides  several  strengths,  online  delivery  of  Fundamental  Level  One  4MAT  training  had  its 
shortcomings.  The  issue  of  no  face-to-face  interactions  did  bother  some  individuals.  The  isolation  of 
reading  and  working  alone  existed  and  in  some  cases,  created  frustrations  c onceming  the  use  of  various 
technologies,  difficulties  with  some  CD  applications,  and  general  technological  “glitches.”  In  the  same 
token,  e -mail  correspondence  and  responses  between  instructors  and  students  alike  were  expected  to  be 
“instant”  because  of  the  speed  of  e-mail  technology.  Anything  less  than  this  speed  of  response  created 
negative  impressions.  For  some  students,  a web-based  course  tends  to  work  against  developing 
personal  relationships  with  classmates  since  e-mail  and  forum  discussion  taketime  to  transcribe  and  to 
communicate  with  others. 

Implications  of  Online  Staff  Development 

The  conversion  of  the  traditional  Fundamental  Level  One  4MAT  training  to  an  online  delivery 
was  considered  successful.  Much  of  this  success  was  due  to  the  already  apparent  structure  of  the  onsite 
training  program  designed  by  Aboutleaming,  Inc.  which  centered  around  learning  styles  and  brain- 
based  research.  All  that  was  necessary  for  the  online  designers  was  to  employ  appropriate 
technological  tools  for  an  effective  online  delivery.  The  virtual  online  course  can  address  and  retain 
the  integrity  of  the  staff  development  training  if  the  technological  tools  are  appropriate  and 
implemented  properly. 

The  National  Board  for  Professional  Teaching  Standards  (NBPTS)  has  designed  not  only 
certification  standards  for  accomplished  teachers  but  it  has  also  been  looking  at  teacher  growth  over 
one’s  professional  life  span.  (Diez  & Blackwell,  1999).  Among  the  NBPTS  charges  is  a 
recommendation  to  design  graduate  teacher  programs  with  three  critical  factors  which  promote  teacher 
growth:  (1)  reflection  on  practice,  (2)  systematic  inquiry  into  practice,  and  (3)  collaboration  with 
others  in  meeting  learner’s  needs.  Reflection  on  practice,  as  cited  by  the  NBPTS,  is  a teacher’s  ability 
to  explain  what  they  do  and  why  they  do  certain  things  in  their  classrooms.  For  example,  online  forum 
discussions  allowed  all  teachers  time  to  reflect  and  express  well-thought  out  responses.  Challenging 
assignments  and  online  discussion  allows  for  the  observance  of  diversity  among  teaching  practices. 
Systematic  inquiry  into  practice  is  illustrated  by  the  emphasis  in  the  online  4MAT  training  to 
incorporate  learning  style  and  left-right  brain  research  into  one’s  lesson  planning.  This  quality  was 
enhanced  by  an  online  lesson  bank  showing  a variety  of  applications  utilizing  such  research 
information.  Collaboration  with  others  in  meeting  learner’s  needs  was  accomplished  by  the  diversity 
of  teachers  from  different  geographical  regions  and  school  settings,  using  such  technological  tools  for 
lesson  critiquing  through  partner  e-mail,  live  chats  on  various  teaching  issues,  and  forums  on  best 
practices. 

As  Landeck  (2000,  p.42)  states,  “To  meet  the  different  learning  styles  and  levels  of  tech  expertise 
of  their  staff,  many  educational  institutions  are  looking  at  a combination  of  online  and  face-to-face 
instruction.”  Online  delivery  systems  can  offer  attractive  educational  and  economical  options  to  school 
inservice  formats.  Hybrid  options  need  to  be  explored  where  onsite-online  combinations  can  be 
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structured  to  fit  individual  school  schedules,  help  to  resolve  critical  need  areas  within  the  educational 
framework,  and  encourage  cooperative  collaborations  on  school  staffs.  In  addition,  hybrid  options  can 
alleviate  administrative  decisions  involving  financial  issues  concerning  substitute  teachers  and  teacher 
time  away  from  the  classroom,  which  inevitably  effects  student  learning  and  security. 

The  time  has  come  to  consider  online  experiences  as  a viable  option  in  professional 
development.  Using  today’s  technological  tools  incorporated  into  relevant,  topical  inservices  for 
teachers,  can  not  only  insure  an  effective  and  economical  delivery  method  of  enhancing  academic  and 
pedagogical  knowledges  but  also  showcase  the  use  of  various  technological  tools  to  enrich  the  learning 
environments  of  students. 
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Abstract 

This  project  is  the  development  of  and  the  evaluation  of  an  online  educational  book  review 
site.  Current  book  reviews  cannot  be  published  in  a timely  fashion.  Usually,  a single  review 
is  published  representing  the  views  of  a single  individual.  Focusing  on  these  two  flaws, 
Online  Educational  Book  Review  Group  publishes  book  reviews  shortly  after  the  book  is 
available;  it  is  a site  that  has  feature  reviews  from  a variety  of  perspectives,  such  as  peer 
evaluations,  invited  reviews  from  well-known  scholars,  authors,  publishers,  and  a discussion 
board  area  for  anyone  who  wishes  to  make  comments.  It  is  operated  entirely  through  a web- 
based  database-driven  design  that  supports  the  submission,  review,  editing,  and  publication 
processes  online. 

Introduction 

Knowledge  construction  provides  an  ideal  learning  environment.  Maurer  and  Davidson 
(1998)  state:  “The  ultimate  purpose  of  instructional  technology’s  support  of  an  exciting,  quality 
learning  curricula  explicitly  is  to  encourage  children  to  discover  basic  data  (facts),  to  integrate  and 
interpret  those  data  into  information,  and  to  apply  that  information  into  action  to  create  knowledge  (p. 
4).”  The  current  design  of  many  online  instructions  features  the  distribution  of  static  information,  or  a 
linear  approach.  This  does  not  provide  an  opportunity  for  learners  to  pursue  critical  thinking  a 
construct  their  own  knowledge.  Frequently,  instructors,  or  content  experts  deliver  the  information,  and 
the  learners  are  denied  an  opportunity  to  contribute.  This  type  of  instruction  is  not  individualized  it  is 
canned  knowledge  that  is  teacher-centered.  Many  instructions  provide  a long  list  of  information 
without  a search  capability.  Even  when  a search  function  is  available,  the  search  capability  lacks  focus, 
like  Internet  search  engines  and  is  a very  time  consuming  process.  Online  bulletin  boards  and  forums 
have  been  considered  and  implemented  as  knowledge  construction  learning  environments.  However, 
bulletin  boards  and  forums  only  provide  for  the  simple  exchange  of  ideas.  It  ignores  the  sophisticated 
human  thinking  process  and  does  not  generate  greater  knowledge  construction. 

There  are  at  least  two  disadvantages  surrounding  the  current  book  review  process:  a)  The 
reviews  are  not  published  in  a timely  fashion  and  b)  there  is  no  provision  to  support  interaction. 
Publishing  a book  may  take  six  to  twelve  months  and  with  the  current  review  process  one  to  two  years 
may  have  elapsed  before  a book  review  is  published.  The  current  schema  only  provides  for  the 
judgment  of  a single  individual,  without  provision  for  the  opinions  of  different  people  or  an 
opportunity  for  interaction.  The  Online  Educational  Book  Review  Project  is  being  formed  to  address 
the  problems  with  the  current  book  review  process  and  to  foster  the  formation  of  an  authentic 
knowledge  construction  community. 

Implementation  Of  Knowledge  Construction 

The  Internet  has  become  a primary  source  of  daily  information  and  possesses  the  potential  for 
knowledge  construction.  An  enormous  amount  of  information  is  presented  to  us  everyday.  The 
Internet  and  WWW  have  been  used  for  the  presentation  and  delivery  of  information.  Obtaining 
information  has  never  been  an  extremely  critical  issue  for  people,  particularly  in  a learning 
environment.  Presently  there  is  an  avalanche  of  information  that  is  constantly  bombarding  the 
populace.  The  shear  excess  is  overwhelming  and  difficult  for  an  ordinary  human  being  to  process. 
Shenk  ( 1 999)argued,  “The  Web  right  now  is  nothing  more  than  a repository  of  convenience.”  (p.  22) 
This  phenomenon  also  occurs  in  an  online  learning  environment  and  poses  a management  problem  for 
both  learners  and  instructors  (Tu,  2000).  Current  online  instructional  designs  rarely  support  a learning 
style  that  engenders  knowledge  construction  and  fails  to  reflect  the  complexities  of  human  thinking. 
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Shenk  (1999)wamed  that  users  should  be  careful  not  to  let  the  Internet  govern  the  way  they  think. 
Frequently  online  communications  occur  to  enable  the  simple  exchange  of  ideas  but  the  threading  that 
occurs  in  an  online  bulletin  board,  for  example,  does  not  satisfy  the  complexity  of  human  thinking. 
Learners  lack  the  resources  to  determine  the  accuracy,  timeliness,  and  social  context  of  the  information 
presented  to  them.  These  facts  about  the  information  received,  accuracy,  timeliness  and  social  context, 
are  critical  to  the  process  of  knowledge  construction. 

Selective  multiple  resources  play  an  important  role  in  the  knowledge  construction  process. 
An  ideal  learning  environment  engages  students  in  deep  thinking,  provides  multiple  viewpoints, 
supports  reflection,  and  offers  frequent  feedback  and  guidance  toward  a higher  standard.  For  example, 
before  reading  or  buying  a book,  people  will  normally  gather  information  from  different  resources, 
such  as  their  friends,  colleagues,  or  relatives,  “word-of-mouth”  or  “appeals  to  authority,”  or 
information  may  be  received  from  multiple  mass  media  reporting  agencies.  After  gathering  or 
receiving  information  from  multiple  resources,  people  will  question,  filter,  claim,  summarize, 
synthesize,  predict,  and  generate  a new  set  of  knowledge.  For  example,  in  the  class  setting, 
recommended  teachers  provide  texts  and  references  in  papers  can  help  to  filter  the  useful  information 
from  the  useless  information  that  may  be  provided  from  friends  and  colleagues. 

Architecture  Of  The  Internet 

The  architecture  of  Internet  information  delivery  does  not  support  information  structure  in  a 
meaningful  way.  Therefore,  developing  tools,  which  foster  self-improving  and  knowledge  construction 
communities  is  very  important.  We  should  move  beyond  a forum  for  simply  exchanging  ideas  and 
opinions,  to  structures,  which  rapidly  capture  and  reflect  knowledge  and  foster  rapid  accumulation  and 
growth  of  a community’s  forces.  In  a learning  community,  one  has  the  chance  to  make  contributions 
and  each  contribution  should  generate  values  and  more  chances  for  knowledge  construction.  Learner’s 
contribute  and  quickly  find  the  best  resources  and  opportunities  to  negotiate  which  items  are  key  to 
knowledge  mining  and  knowledge  construction.  Online  database,  XHTML,  XML,  and  artificial 
intelligent  (AI)  are  promising  technologies  to  support  knowledge  construction  environment. 

Theoretical  Framework 

Automated  Collaborative  Information  Filtering 

Automated  Collaborative  Filtering  of  information  (ACF)  operates  for  the  distribution  of 
opinions  and  ideas  in  society  and  facilitates  contacts  between  people  with  common  interests 
(Chislenko,  1997).  ACF  enhances  existing  mechanisms  of  knowledge  distribution,  increases  their 
efficiency,  optimizes  knowledge  processing  in  the  society,  and  accelerates  the  evolution  of  ideas  in 
practically  all  subject  areas;  and,  also,  provides  a superior  tool  for  information  retrieval  systems  that 
facilitate  users’  knowledge  construction  in  a meaningful  and  personalized  way.  As  an  artificial  system 
that  integrates  and  processes  knowledge  of  multiple  human  participants,  ACF  represents  an 
intermediate  stage  between  human  and  purely  artificial  intelligence  and  lays  the  foundation  for  the 
future  knowledge  construction. 

People  must  possess  both  the  necessary  general  knowledge  and  the  special  information 
relevant  to  their  particular  situation  to  make  efficient  personal  selections.  The  collection  of  necessary 
information  requires  that  one  determine  which  items  deserve  their  attention.  This  determination 
requires  the  exchange  of  personal  experiences  among  individuals  and  the  sharing  of  personalized 
advice  on  specific  issues.  When  someone  is  confronted  by  the  necessity  of  making  a decision  about  an 
unknown  situation,  they  can  ask  advice  of  their  friends,  and  follow  their  suggestions.  Here,  one’s 
circle  of  acquaintances  effectively  plays  the  role  of  an  information  filter,  designating  the  most  relevant 
options  and  providing  leads  for  further  exploration. 

Recker  and  Lawless  (1999)  noted  the  potentials  of  implementing  ACF  into  a learning 
environment.  They  argued  that  three  issues  are  important  in  implementing  ACF  into  instruction:  a) 
Information  about  who  contributes  opinions  is  as  important  as  the  contribution  itself,  b)  because 
contributors  form  a community,  they  have  their  own  language  and  set  of  values;  c)  evaluating 
information  sources  and  contributing  rating  is  a metacognitive  activity. 

Potential  of  an  Online  Database  for  Knowledge  Construction 

Online  databases  provide  opportunities  to  facilitate  knowledge  construction  or  knowledge 
mining.  Learners  are  required  to  perform  intentional  searches  and  the  information  obtained  is 
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contexturalized  and  personalized.  This  process  encourages  meaningful  and  higher-order  thinking. 

Student  publishing  and  peer  evaluation  that  occur  on  an  online  database  (Tu,  1999)  propagate 
improved  quality  in  students’  work.  Jonassen,  Davidson,  Collins,  Campbell,  andHagg  (1995)  argued 
that  reflecting  upon  knowledge,  stating  learning  intentions,  and  publishing  ideas  to  a communal 
database  produce  cumulative,  progressive  results  for  the  community/group. 

Online  Educational  Book  Review  Project 

The  Online  Educational  Book  Review  Project  is  an  Internet  web  site  community  that  provides 
a forum  for  education  book  reviews  by  authors,  publishers,  professional  book  reviewers,  and  regular 
consumers.  It  offers  each  an  equal  opportunity  to  contribute  their  professional  opinions. 

To  enter  the  online  community,  users  are  required  to  create  a new  account  to  become  a 
community  member  and  provide  basic  profiles  that  can  be  accessed  by  other  community  members.  A 
logging  on  process  is  required  when  a member  accesses  the  community.  The  open  page  working  as 
information  agent  is  customized  to  provide  members  with  news,  new  review  messages,  and 
recommending  activities  according  members’  profiles  etc. 

Members  can  perform  a search  function  to  identify  books  and  book  review  messages.  The 
messages  can  be  retrieved  linearly  or  non-linearly.  Members  can  c ontribute  book  review  messages  by 
completing  the  online  form  with  fields  for  message  title,  keywords,  and  rating  etc.  All  of  procedures 
are  performed  on  the  browser  and  access  is  designed  to  be  user  friendly. 

This  knowledge  construction  community  for  book  review  is  a learning  community  for  anyone 
who  is  interested  to  learning  about  educational  books.  The  target  audience  is  composed  of  anyone  who 
is  interested  in  certain  books  or  would  like  to  discuss  certain  books.  In  phase  one,  classroom  teachers 
can  integrate  this  knowledge  construction  community  into  their  class  reading  and  reading 
report/summary/critique  assignments.  The  book  topics  range  from  educational  philosophical  content  to 
children’s  storybooks.  A special  arrangement  can  be  made  to  accommodate  class  reading  and  report 
projects  to  a classroom  community. 

Conclusions 


The  Online  Educational  Book  Review  Project  with  web-based  database  utilization  enhances 
an  information  filtering  learning  environment,  supports  the  complex  human  information  process,  and 
fosters  authentic  knowledge  construction.  It  is  able  to  capture  and  reflect  knowledge  and  foster 
accumulation  and  growth  of  a community's  forces.  This  concept  can  also  be  implemented  to  enable 
learners  and  instructors  to  find  and  form  collaborative  teams  around  mutual  goals,  skills,  and 
processes.  New  technologies,  such  as  XHTML , XML  and  artificial  intelligence,  can  be  applied  to  the 
online  database  in  the  future. 
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Unlocking  the  gates  to  the  kingdom: 
Designing  web  pages  for  accessibility 


Steven  C.  Mills 
University  of  Kansas 

Although  the  use  and  availability  of  information  resources  on  the  Internet  have  skyrocketed 
over  the  last  few  years,  cyberspace  has  predominantly  been  a domain  of  persons  without  physical  or 
mental  disabilities.  While  the  Internet  is  easily  accessible  to  individuals  without  disabilities,  for  all 
practical  purposes  the  gates  to  cyberspace  have  been  closed  and  locked  for  persons  with  disabilities. 
According  to  Tim  Berners-Lee,  World  Wide  Web  Consortium  Director  and  inventor  of  the  World 
Wide  Web,  “ The  power  of  the  Web  is  in  its  universality . Access  by  everyone  regardless  of  disability  is 
an  essential  aspect 

The  development  of  web  pages  that  exclude  persons  with  disabilities  is  unfortunate  because 
these  individuals  often  have  the  most  to  gain  from  Internet  technology.  Access  to  Internet  technology 
has  the  potential  to  increase  or  enhance  the  productivity  and  independence  ofpersons  with  disabilities. 
In  fact,  access  to  the  Web  may  be  more  critical  for  individuals  with  disabilities  than  for  the  general 
population  who  can  access  information  resources  using  conventional  del  ivery  systems.  For  example: 

• Persons  with  disabilities  may  be  location-bound.  These  individuals  could  use  the  Internet  to  shop 
for  almost  anything,  research  health  questions,  participate  in  on-line  discussions,  and  contact 
friends  and  family. 

• Persons  who  are  blind  may  wait  indefinitely  for  information  to  be  made  available  in  Braille  or 
audiotape  formats.  These  individuals  could  use  the  Internet  to  acquire  access  to  the  same 
information  at  about  the  same  time  it  is  available  to  persons  who  are  sighted. 

• Persons  who  are  unable  to  hold  a pen  or  use  a mouse  or  keyboard  could  use  speech  recognition 
software  to  accomplish  the  daily  routines  and  tasks  of  life. 

While  it  is  difficult  to  identify  who  uses  the  Internet,  a variety  of  organizations  and  methods 
have  been  used  to  capture  data  that  defines  the  audience  of  Internet  users.  Kaye  (2000)  analyzed 
national  survey  data  to  determine  computer  and  Internet  use  among  people  with  disabilities  to 
determine  that  people  with  disabilities  have  considerably  less  access  to  the  Internet  when  compared  to 
people  with  no  disabilities  — 1 1 .4%  versus  31.1  %.  Kaye  concluded  that  while  people  with  disabilities 
have  the  most  to  gain  from  Internet  technology,  the  potential  benefits  of  this  technology  are  far  from 
being  realized.  According  to  Kaye  the  problem  is  one  of  access  that  is  a result  of  limited  ownership  of 
computer  technology  and  specialized  software  by  persons  with  disabilities  and  the  lack  of  user 
interfaces  that  encourage  use  of  the  technology  among  people  with  disabilities. 

The  Georgia  Tech  Graphics,  Visualization  and  Usability  Center  (GVU)  conducts  a large  scale 
survey  in  April  and  October  of  each  year  (available  at  http://www.gvu.gatech.edu/  user_surveys/). 
Beginning  with  the  second  survey  conducted  in  October,  1 994,  respondents  were  asked  about  their 
disability  status.  In  October,  1 994,  5. 1 1 % of  the  respondents  indicated  they  had  a disability.  The  tenth 
and  latest  survey  conducted  in  October,  1998,  indicated  that  the  proportion  of  respondents  with 
disabilities  increased  to  7.68%.  In  all  GVU  surveys  the  Vision  category  was  identified  by  over  half  the 
respondents  who  indicated  they  have  a disability.  Although  this  survey  confirms  the  incidence  of 
Internet  use  among  people  with  disabilities  is  low  (approximately  7% -8%)  when  compared  to  the 
general  population,  these  low  numbers  may  be  a self-fulfilling  indication  that  people  with  disabilities 
do  not  have  access  to  the  Internet  or  have  difficulties  in  accessing  information  on  the  World  Wide 
Web. 


Web  pages  often  do  not  allow  for  the  eventuality  that  some  web  surfers  may  not  be  able  to 
see,  hear,  move,  or  process  some  forms  of  information.  Other  web  surfers  may  have  difficulty  reading 
or  comprehending  text  or  may  not  be  able  to  use  a mouse  or  keyboard.  Many  experienced  web 
designers  and  authors,  however,  are  completely  unaware  of  accessibility  issues  and,  therefore,  have 
little  or  no  experience  in  making  their  web  pages  usable  by  persons  who  cannot  see  the  screen  or  use  a 
mouse  the  same  way  that  the  web  page  authors  do.  They  have  what  Bartlett  ( 1 999)  calls  a “skewed 
mindset”  in  which  they  develop  web  pages  to  convey  content  visually.  Jane  Jarrow,  president  of 
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Disability  Access  and  Information  Support,  explains  that  there  is  an  art  to  making  web  pages  more 
accessible  “but  people  don’t  think  to  do  it.”  (Camevale,  1999). 

Adjusting  browser  preferences  and  using  assistive  technologies  may  lower  the  access  barriers 
to  web  pages,  but  the  best  method  for  providing  equal  accessibility  is  by  building  accessibility  features 
into  the  web  site  itself  (Casey,  1 999).  Much  of  what  web  page  authors  can  do  to  make  web  pages  more 
accessible  is  relatively  simple  and  can  be  achieved  through  proper  design  strategies  using  the 
accessibility  features  of  HTML  4.01  and  other  web  authoring  resources.  Web  pages  that  are  accessible 
to  people  with  disabilities  are  highly  accessible  to  everyone.  Thus,  web  accessibility  is  a design  issue. 
Design  strategies  that  create  accessible  web  pages  also  facilitate  the  creation  of  well-designed  web 
pages.  Improvements  to  a web  page  or  site  that  enable  web  surfers  with  disabilities  to  access  it  also 
improve  the  web  page  for  all  surfers. 

The  World  Wide  Web  Consortium  (W3C)  has  recognized  a disparity  in  accessibility  to  the 
Web  between  persons  with  and  without  disabilities  and  has  responded  by  developing  a set  ofweb  page 
design  standards  that  specifically  address  the  issue  of  web  accessibility.  Web  accessibility  is  based  on 
design  principles  that  provide  for  the  development  of  web  pages  to  accommodate  the  needs  of  a broad 
range  ofusers,  computers,  and  telecommunications  systems  without  regard  to  age  or  disability.  When  a 
web  site  is  accessible,  anyone  browsing  the  site  should  be  able  to  gain  a complete  understanding  of  the 
information  presented  on  the  site  as  well  as  have  an  undiminished  ability  to  interact  with  the  site.  The 
W3C  has  embraced  the  issue  of  accessibility  through  its  Web  Accessibility  Initiative  (WAI).  The  WAI 
has  developed  a set  of  guidelines,  the  Web  Content  Accessibility  Guidelines,  for  use  by  web  page 
authors  to  incorporate  accessibility  features  into  the  design  of  web  pages.  The  purpose  of  this  paper  is 
to  describe  webpage  design  principles  and  strategies  based  on  the  WAI’s  guidelines  that  will  facilitate 
the  development  of  web  pages  that  open  the  gates  of  the  virtual  kingdom  to  persons  with  disabilities. 


The  W3c  Web  Accessiblity  Initiative 

Who  Is  the  World  Wide  Web  Consortium? 

In  October  1994,  Tim  Berners-Lee,  inventor  of  the  Web,  founded  the  World  Wide 
Web  Consortium  (W3C)  at  the  Massachusetts  Institute  of  Technology,  Laboratory  for 
Computer  Science  (MIT/LCS).  The  W3C  was  created  to  promote  and  manage  the  evolution 
of  the  Internet  and  to  ensure  its  interoperability.  The  W3C  has  more  than  400  Member 
organizations  from  around  the  world  and  is  financed  by  its  members  and  by  public  funds. 
Membership  in  the  W3C  is  available  to  any  organization.  Along  with  MIT/LCS  in  the  United 
States,  the  W3C  is  jointly  hosted  at  sites  in  France  and  Japan  and  W3C  offices  are  located  in 
1 1 other  countries. 

W3C  activities  and  other  work  are  organized  into  four  domains:  1 ) Architecture  Domain  to 
develop  the  underlying  technologies  of  the  Web;  2)  Technology  and  Society  Domain  to  understand 
ethical  and  legal  issues  from  a new  international  perspective  and  in  light  of  new  technology;  3)  User 
Interface  Domain  to  improve  user  interaction  with  the  Web  including  work  on  formats  and  languages; 
and  4)  Web  Accessibility  Initiative  to  pursue  accessibility  of  the  Web  through  five  primary  areas  of 
work:  technology,  guidelines,  tools,  education  and  outreach,  and  research  and  development.  The  W3C 
has  published  more  than  20  technical  specifications  for  the  Web’s  infrastructure  since  its  inception. 
Each  specification  not  only  builds  on  its  predecessor,  but  is  designed  to  integrate  with  future 
specifications  as  well. 

The  Web  Accessibility  Initiative 

The  Web  Accessibility  Initiative  (WAI)  is  an  official  domain  of  W3C.  The  WAI  works  across 
all  the  other  domains  and  works  internationally  in  all  three  host  sites  of  the  W3C.  The  WAI  is 
sponsored  by  representatives  of  web  development  industries,  disability  organizations,  research 
organizations,  and  government.  Some  of  the  WAI  sponsors  include  the  National  Science  Foundation, 
National  Institute  on  Disability  and  Rehabilitation  Research,  Microsoft,  IBM,  Lotus  Development,  and 
NCR. 
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To  facilitate  the  efforts  of  promoting  Web  accessibility,  the  WAI  joined  forces  with  the  W3C 
HTML  Working  Group  in  the  design  of  HTML  4.0  and  in  December,  1997,  HTML  4.0  became  a W3C 
recommendation  (see  http://www.w3.org/TR/REC-html40/).  In  May.  1998,  an  official  W3C 
recommendation  for  Cascading  Style  Sheets,  Level  2,  (CSS2)  was  issued  (see  www.w3.org/TR/REC- 
CSS2/).  In  May,  1 999,  the  WAI  issued  the  Web  Content  Accessibility  Guidelines  1 .0  ( see 
www.w3.org/TR/WAI-WEBCONTENT/).  These  guidelines  incorporated  the  recommendations  of 
HTML  4.0  and  CSS2  and  were  intended  for  use  by  all  web  content  developers  including  page  authors, 
site  designers,  and  developers  of  authoring  tools. 

Web  Content  Accessibility  Guidelines  1.0 

The  Web  Content  Accessibility  Guidelines  (WCAG)  consist  of  14  guidelines.  Each  of  the 
guidelines  of  the  WCAG  is  comprised  ofmultiple  checkpoints  or  sub-guidelines.  Assigned  to  each  of 
the  checkpoints  is  a priority  level  that  is  based  on  the  checkpoint’s  potential  impact  on  accessibility. 
Priority  1 checkpoints  are  “must  satisfy”  requirements  without  which  some  groups  will  find  it 
impossible  to  access  information  in  a web  page.  Priority  2 checkpoints  are  “should  satisfy” 
requirements  without  which  one  or  more  groups  will  find  it  difficult  to  access  information  in  a web 
page.  Priority  3 checkpoints  are  “may  address”  requirements  without  which  one  or  more  groups  will 
find  it  somewhat  difficult  to  access  information  in  a web  page. 

The  WCAG  provide  for  three  levels  of  conformity  to  the  guidelines:  Level  A,  AA,  and  AAA. 
For  Level  A conformity,  all  Priority  1 checkpoints  are  satisfied;  for  Level  A A all  Priority  2 checkpoints 
are  satisfied;  for  Level  AAA  all  Priority  3 checkpoints  are  satisfied.  Conformance  levels  are 
cumulative.  For  example,  Level  AAA  conformance  would  indicate  that  a web  page  conforms  to 
Priority  1 , 2,  and  3 checkpoints.  Webpages  can  display  logos  to  indicate  a claim  of  conformance  to  a 
specified  level  of  conformity  with  the  WCAG  1.0. 

The  WCAG  are  formulated  around  two  general  web  page  design  strategies:  ensuring  graceful 
transformation  and  making  content  understandable  and  navigable ; 

3.  Ensuring  Graceful  Transformation.  Web  surfers  may  operate  in  contexts  very  different  from  the 
one  in  which  a web  page  is  developed.  Therefore,  webpages  should  transform  gracefully.  A page 
transforms  gracefully  when  it  remains  accessible  despite  any  constraints  that  may  include  (though 
not  be  limited  to)  physical,  sensory,  and  cognitive  disabilities,  work  constraints,  and  technological 
barriers.  For  example,  a web  surfer  may  not  be  able  to  see,  hear,  move,  or  use  a keyboard  or 
mouse,  or  may  have  difficulty  reading  or  comprehending  text.  The  surfer  may  have  a small  screen, 
a slow  Internet  connection,  an  early  version  of  a browser,  a different  browser,  a voice  browser,  or  a 
different  operating  system.  For  web  pages  to  transform  gracefully,  structure  must  be  separate  from 
presentation.  Structure  refers  to  the  logical  organization  of  a page  while  presentation  refers  to  how 
a page  is  rendered,  such  as  print,  computer  graphics,  text,  or  synthesized  speech.  The  theme  of 
graceful  transformation  is  addressed  primarily  by  Guidelines  1 to  1 1 . 

• Making  Content  Understandable  and  Navigable.  Web  page  authors  should  make  page  content 
understandable  and  navigable.  The  language  of  a web  page  should  be  clear  and  simple,  but  also 
provide  understandable  mechanisms  for  navigating  within  and  between  pages.  Providing 
navigation  tools  and  orientation  information  in  webpages  maximizes  accessibility  and  usability. 
Not  all  surfers  can  make  use  of  visual  clues  such  as  image  maps,  proportional  scroll  bars,  side-by- 
side  frames,  or  graphics  that  guide  sighted  users  with  graphical  desktop  browsers.  Web  surfers 
may  also  lose  contextual  information  when  they  can  only  view  a portion  of  a page,  either  because 
they  are  accessing  the  page  one  word  at  a time  as  with  a speech  synthesizer  or  a Braille  display,  or 
one  section  at  a time  as  with  a small  or  magnified  display.  Without  orientation  information,  users 
may  not  be  able  to  understand  very  large  tables,  lists,  or  menus.  The  theme  of  making  content 
understandable  and  navigable  is  addressed  primarily  in  Guidelines  1 2 to  14. 


Designing  Accessible  Web  Pages 

Principles  of  Accessible  Web  Design 
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The  WAI  has  produced  an  extensive  set  of  guidelines  for  authoring  accessible  web  pages.  Since  the 
Guidelines  are  a technical  document  that  may  be  somewhat  overwhelming  to  a beginning  web  author, 
the  HTML  Writers  Guild  (see  www.hwg.org)  has  proposed  six  principles  of  accessible  web  design 
upon  which  the  WCAG  were  written.  The  followingprinciples  are  the  basic  rules  for  accessible  design 
that  formed  the  specific  instances  described  in  each  individual  entry  in  the  guidelines  (Bartlett,  1 998): 

1.  Create  pages  that  conform  to  accepted  standards.  Use  the  W3C  recommendations  for  WCAG, 
HTML  4.01 , and  CSS2  for  designing  web  pages. 

2.  Know  the  difference  between  structural  and  presentation  elements.  For  example,  <EM>, 
<ADDRESS>,  and  <LI>  are  structural  elements  while  <B>  and  <CENTER>  are  presentation 
elements.  Use  HTML  structural  elements  to  convey  page  content  and  style  sheets  to  convey  page 
presentation  and  formatting. 

3.  Use  HTML  4.01  features  to  provide  information  about  the  purpose  and  function  of  elements. 
Attributes  such  as  TITLE  and  CLASS  allow  the  web  author  to  provide  additional  information  on 
the  function  and  meaning  of  particular  tags,  thus  increasing  the  accessibility  of  the  page. 

4.  Ensure  that  pages  can  be  navigated  by  keyboard.  For  example,  using  client-side  image  maps  with 
the  ALT  tag  and  ACCESSKEY  attribute  will  accommodate  keyboard  navigation  by  a browser 
agent. 

5.  Provide  alternative  or  text-based  methods  to  access  non-textual  content  that  includes  images, 
scripts,  multimedia,  tables,  forms,  and  frames  for  user  agents  that  do  not  display  them. 

6.  Be  careful  of  common  programming  techniques  that  can  reduce  the  accessibility  of  your  site  such 
as  ASCII  art,  blinking  text,  or  adjacent  links  that  are  separated  by  non-printable  characters. 

HTML  4.01 

HTML  4.01  is  W3C's  recommendation  for  the  latest  version  of  HTML.  HTML  4.01  was 
released  on  December  24,  1999,  and  fixes  bugs  in  the  HTML  4.0  specification,  which  for  instance, 
omitted  the  name  attribute  on  the  IMG  and  FORM  elements.  HTML  4.01  defines  the  semantics  and 
data  types  for  HTML.  HTML  4.0 1 includes  mechanisms  for  style  sheets,  scripting,  embedding  objects, 
improved  support  for  right  to  left  and  mixed  direction  text,  and  enhancements  to  forms  for  improved 
accessibility  for  people  with  disabilities.  HTML  4.01  is  specified  according  to  three  variants: 

1.  HTML  4.0 1 Strict  excludes  the  presentation  attributes  and  elements  that  the  W3C  expects  to 
phase  out  as  support  for  style  sheets  matures.  Web  authors  should  use  the  Strict  DTD  when 
possible,  but  may  use  the  Transitional  DTD  when  support  for  presentation  attribute  and 
elements  is  required. 

2.  HTML  4.01  Transitional  includes  presentation  attributes  and  elements  that  W3C  expects  to 
phase  out  as  support  for  style  sheets  matures.  Web  authors  should  use  HTML  4.0 1 Strict  when 
possible,  but  may  use  Transitional  when  support  for  presentation  attributes  and  elements  is 
required 

3.  HTML  4.01  Frameset  is  used  for  documents  with  frames.  This  variant  is  identical  to  the 
HTML  4.0 1 Transitional  except  for  the  content  model  of  the  "HTML"  element:  in  frameset 
documents,  the  "FRAMESET"  element  replaces  the  "BODY"  element. 

The  web  author  designates  which  of  these  variants  are  used  on  a web  page  by  inserting  a line 
called  a Document  Type  Definition  (DTD)  at  the  beginning  of  the  document.  This  line  is  used  by  the 
validation  service  to  determine  the  variant  of  HTML  4.0  that  is  used  on  a page.  Each  variant  has  its 
own  DTD.  For  example,  the  DTD  for  a web  page  that  is  HTML  4.01  Transitional  is: 

<!DOCTYPE  HTML  PUBLIC  "-//W3C//DTD  HTML  4.01  Transitional//EN" 
"http://www.w3.org/TR/html4/loose.dtd"> 

The  complete  HTML  4.0 1 specification  is  available  in  English  in  several  formats,  including 
HTML,  plain  text  Postscript,  and  PDF  at  http://www.w3.org/TR/1999/REC-html401-1999l224. 

Cascading  Style  Sheets 

Cascading  style  sheets  (CSS)  facilitate  accessibility  to  web  pages  by  separating  document  structure 
from  presentation.  CSS2  is  the  current  specification  for  cascading  style  sheets  and  is  a recommendation 
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of  the  W3C.  A discussion  of  the  accessibility  features  of  CSS2  may  be  found  at 
www.w3.org:80/TR/CSS-access. 
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Style  sheets  were  designed  to  allow  precise  control  over  page  presentation  properties  such  as  character 
spacing,  text  alignment,  object  position  on  the  page,  audio  and  speech  output,  or  font  characteristics 
apart  from  markup.  By  separating  style  from  content,  web  authors  can  simplify  the  HTML  in  their 
documents  while  making  the  documents  more  accessible  at  the  same  time.  CSS  facilitates  accessibility 
in  several  ways: 

• Tag  misuse.  CSS2  allows  precise  control  over  spacing,  alignment,  and  positioning,  thus, 
eliminating  the  practice  of  misusing  a structural  element  for  stylistic  effects.  For  example,  the 
BLOCKQUOTE  and  TABLE  elements  in  HTML  are  intended  to  mark  up  quotations  and  table 
data,  but  are  frequently  used  to  create  visual  effects  such  as  indentation  and  alignment. 

• Image  misuse.  CSS2  positioning  properties  eliminate  the  need  for  invisible  images  to  position 


• Font  control.  CSS2  provides  for  precise  control  over  font  size,  color,  and  style. 

• User  override.  CSS2  allows  users  to  override  author  styles  and  to  view  documents  with  their  own 
preferred  fonts,  colors,  and  styles  by  specifying  them  in  a user  style  sheet. 

• Orientation  and  navigation.  CSS2  provides  support  for  automatically  generated  numbers, 
markers,  and  other  content  that  assists  surfers  in  staying  oriented  within  a document.  Long  lists, 
tables,  or  documents  are  easier  to  navigate  when  numbers  or  other  contextual  clues  are  provided  in 
an  accessible  manner. 

• Aural  style  sheets.  Aural  style  sheets  are  used  to  specify  how  a document  sounds  when  rendered 
as  speech.  Aural  style  sheets  allow  authors  and  users  to  specify  properties  such  as  the  volume  of 
spoken  content,  background  sounds,  and  spatial  properties  for  sound  that  can  add  effects  to 
synthesized  speech.  These  effects  correspond  with  those  achieved  with  styled  fonts  for  visual 
output. 

Style  sheets  minimally  should  provide  declarations  for  all  structural  elements  used  in  the 
HTML  source.  Web  pages  using  CSS  essentially  need  to  be  unadorned  or  unformatted.  Tocreateaweb 
page  with  CSS,  use  HTML  code  with  few  or  no  deprecated  HTML  tags.  Deprecated  tags  are  HTML 
tags  that  are  part  of  the  HTML  specification  but  are  expected  to  be  phased  out  of  subsequent  versions 


Style  declarations  can  be  embedded  at  the  beginning  of  an  HTML  document  using  a STYLE 
tag  (e.g.,  <STYLE>  </STYLE>)  or  embedded  inside  elements  in  HTML  (called  inline  styles)  using  a 
STYLE  attribute  (e.g.,  <H1  STYLE- ’text  align:  center”>Heading  Level  One</Hl>).  Style  sheets 
external  toHTML  documents  can  be  linked  to  these  documents.  All  methods  of  style  declarations  can 
be  used  in  a single  web  page. 

Validation  Services 

BOBBY.  Bobby  (http://www.cast.org/bobby/)  is  a web-based  tool  that  analyzes  web  pages  for 
accessibility  according  to  the  WCAG.  Bobby  is  offered  as  a free  public  service  by  the  Center  for 
Applied  Special  Technology  (http://www.cast.org)  to  facilitate  its  mission  to  expand  opportunities  for 
people  with  disabilities  through  the  innovative  uses  of  computer  technology.  Although  Bobbyisanon- 
line,  web -based  validation  service,  Bobby  also  requires  a manual  examination  of  those  components  of  a 
web  page  that  Bobby  cannot  examine  automatically. 

Bobby's  analysis  of  accessibility  is  based  on  the  W3C’s  Web  Content  Accessibility  Guidelines.  The 
combination  of  automatic  and  manual  examination  of  a web  page  makes  Bobby  Approved  status 
equivalent  to  WAI  Conformance  Level  A— that  is,  all  Priority  1 items  have  passed. 

For  example,  to  become  Bobby  approved,  a Web  site  must: 

6.  provide  text  equivalents  for  all  non-text  elements  (i.e.,  images,  animations,  audio,  video) 

7.  provide  summaries  of  graphs  and  charts 

8.  ensure  that  all  information  conveyed  with  color  is  also  available  without  color 
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9.  clearly  identify  changes  in  the  natural  language  of  a document's  text  and  any  text  equivalents  (e.g., 
captions)  of  non-text  content 

10.  organize  content  logically  and  clearly 

1 1.  provide  alternative  content  for  features  (e.g.,  applets  or  plug-ins)  that  may  not  be  supported 

Bobby  also  analyzes  web  pages  for  compatibility  with  various  browsers.  Analysis  is  based  on 
documentation  from  browser  vendors  when  available.  Bobby  automatically  checks  sites  for 
compatibility  with  HTML  4.0.  For  accessibility  and  tag  compatibility  with  browser  specifications  other 
than  HTML  4.0,  use  the  Advanced  Options  of  Bobby.  Once  a web  site  receives  a Bobby  Approved 
rating,  it  is  entitled  to  use  a Bobby  Approved  icon  on  its  web  pages.  Additionally,  a Bobby  approved 
web  site  is  eligible  for  listing  in  the  Bobby  Approved  Database  at 

http://dev.cast.org/bobby/approved_database.cfm.  This  database  may  draw  positive  attention  to  a Web 
site  and  help  others  understand  that  Web  accessibility  is  an  important  initiative. 

Bobby  is  only  one  step  in  the  process  of  making  a site  accessible  to  as  many  people  as 
possible.  CAST  recommends  that  web  developers  use  Bobby  as  a first  step  to  ensure  accessible  web 
page  design.  Other  web-based  tools  are  available  to  analyze  web  pages  for  their  conformance  to  the 
accessibility  guidelines,  HTML  4.01,  and  CSS2: 

1.  HTML  4.01.  To  validate  a web  page  for  HTML  4.0,  the  page  must  contain  a Document  Type 
Definition  (DTD)  at  the  beginning  of  the  page.  The  validation  tool  knows  which  variant  of  HTML 
4.01  is  being  validated  based  on  the  DTD.  The  HTML  validator  can  be  accessed  at 
http://validator.w3.org/.  The  HTML  validator  provides  validation  by  URL  or  by  uploading  HTML 
into  the  validator.  Web  authors  can  validate  the  HTML  used  in  the  web  pages  for  conformancy 
with  the  HTML  4.01  recommendation,  and  web  pages  that  validate  can  display  a logo  to  identify 
the  conformance  claim. 

2.  CSS2.  A validator  for  CSS2  can  be  downloaded  from  http://jigsaw.w3.org/css-validator/  or 
validated  by  URL,  by  entering  CSS  text  into  the  validator,  or  by  uploading  CSS  text  into  the 
validator.  Validating  style  sheets  requires  the  use  of  valid  HTML.  Web  authors  can  validate  the 
style  sheets  used  in  the  web  pages  for  conformancy  and  style  sheets  that  validate  can  display  a 
logo  to  identify  the  conformance  claim 

Validation  Procedures 

Validation  of  accessibility  is  a continuous  process.  Validation  of  accessibility  should  be 
performed  with  both  automatic  tools  and  manual  examination.  Validation  procedures  should  be 
followed  even  at  the  earliest  stages  of  web  page  development  where  accessibility  issues  are  easier  to 
identify,  correct,  or  avoid.  To  assist  in  web  page  design  and  validation,  the  W3C  provides  a Checklist 
of  Checkpoints  for  Web  Content  Accessibility  Guidelines  1 .0  at  http://www.w3.Org/TR/l  999/WAI- 
WEBCONTENT-1 9990505/full-checklist. html.  A table  containing  a checklist  with  the  Priority  1 
Guidelines  is  provided  at  the  end  of  this  article. 

When  developing  web  pages,  the  following  validation  methods  will  facilitate  the  development 
of  accessible  web  pages  that  conform  to  the  WCAG: 

9.  Use  an  automated  validation  tool  and  browser  validation  tool  but  remember  that  automated  tools 
do  not  address  all  accessibility  issues. 

1 0.  Validate  source  syntax  (HTML,  XML,  etc.). 

1 1.  Validate  style  sheets  (CSS2,  etc.). 

1 2.  Test  web  pages  with  a text-only  browser. 

1 3.  Test  web  pages  with  multiple  graphics  browsers  and  browser  versions. 

14.  Test  web  pages  with  browser  or  screen  reader. 

1 5.  Test  web  pages  with  multiple  screen  resolutions. 

1 6.  Test  web  pages  with  spelling  and  grammar  checkers 
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The  following  web  page  authoring  tips  provide  a quick  guide  for  establishing  WAI  Level  A 
conformity  (see  checklist  at  www.w3.org/TR71999/WAI-WEBCONTENT-l  9990324/full- 
checklist. htmland)  and  Bobby  Approval  of  a web  page.  A more  comprehensive  discussion  of 
techniques  that  implement  the  checkpoints  is  defined  in  the  WCAG  at  http://www.w3.org/TR7WAI- 
EBCONTENT-TECHS/.  Additionally,  The  W3C  has  developed  and  published  a downloadable 
Curriculum  for  Web  Content  Accessibility  Guidelines  1.0  slide  set  at 

http://www.starlingweb.com/wai/wcag/.  Several  of  the  following  examples  are  provided  in  the  example 
set  of  this  curriculum  at  http://www.starlingweb.com/wai/wcag/oversam.htm  (Chuck  Letourneau  & 
Geoff  Freed,  Copyright  © 2000  W3C. 

Using  Images  in  a Web  Page 

• Provide  cl  ient-side  image  maps  instead  of  server-side  image  maps  except  where  the  regions  cannot 
be  defined  with  an  available  geometric  shape.  When  using  image  maps,  use  the  ALT  tag  for  each 
image  and  image  map  link  and  provide  a text  version  of  the  links  of  an  image  map  elsewhere  on 
the  web  page.  For  a cl  ient-side  image  map,  describe  the  destination  to  which  each  active  area  will 
link.  For  example: 

<A  HREF-1.. /index. html"> 

<IMG  SRC="../gifs/logo.gif'  WIDTH=,,630n  HEIGHT-' 1 11"  ALIGN-'top"  border=MOu 
naturalsizeflag="3"  ALT-’Graphic  Header  Image  with  Mapped  Links  to  other  pages" 
USEMAP="#insideheaderb55d8d01"> 

<MAP  NAME="logo55d8d01"> 

<AREA  HREF="newsletter.htmr  COORDS=”534,85,625,108M  SHAPE="rect" 
ALT=,,Link  to  Newletter  Page"> 

<AREA  HREF-'staff.html"  COORDS=,,462,86,533,108,,  SHAPE^’rect" 
ALT="Link  to  Staff  Page"> 

<AREA  HREF-'syscomp.htmr  coords=”  100,85, 193, 108”  SHAPE-’rect" 
ALT=MLink  to  System  Components  Page"> 

</MAP> 

</A 

Bitmapped  text  images  cannot  be  read  by  a screen  reader  and  should  also  use  the  ALT  tag.  For 
example, 

<IMG  SRC="wai-lg.gif'  ALT="Graphical  Link  to  Web  Accessibility  Initiative’^ 

Images  or  buttons  with  links  should  be  large  enough  to  allow  surfers  who  use  an  alternate  type  of 
pointing  device  with  their  computer  to  easily  select  the  image. 

Using  Tables  in  a Web  Page 

a.  Use  tables  primarily  to  convey  statistical  data  or  organized  information.  For  data  tables  that  have 
two  or  more  logical  levels  of  row  or  column  headers,  use  markup  to  indicate  data  cells  <TD>  and 
header  cells  <TH>. 

b.  Restrict  the  use  of  tables  for  layout  of  web  pages.  Use  Cascading  Style  Sheets  to  layout  and  format 
text  and  images  on  a web  page  instead  of  tables.  If  a table  is  used  for  formatting  text,  use  the 
SUMMARY  tag.  For  example: 

<TABLE  width=”640”  bordei=”0”  CELLSPACING=,,0"  CELLP  ADDING-^" 

SUMMARY- 'This  table  is  for  formatting  purposes  only."> 

c.  If  tables  are  used  for  formatting  and  placement,  test  them  in  a text-only  browser  such  as  Lynx  to 
verify  that  a text-only  browser  will  display  your  content  properly  or  with  a browser  reader  to 
determine  that  the  layout  of  your  web  page  is  comprehensible  to  users  of  assistive  technology. 

Using  Frames  in  a Web  Page 

• Title  each  frame  in  a web  page  to  facilitate  frame  identification  and  navigation.  For  example: 
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<FRAME  SRC="tocleft.htmr  TITLE="Table  of  Contents  Frame"> 


• If  possible,  do  not  use  frames  in  a web  page  because  frames  cause  difficulty  in  printing,  viewing, 

and  navigation  for  all  users,  not  just  those  with  physical  disabilities. 

Using  Applets  and  Scripts  in  a Web  Page 

a.  Ensure  that  pages  are  u sable  when  scripts,  applets,  or  other  programmatic  objects  are  turned  off  or 
not  supported. 

b.  If  pages  are  not  usable  when  turning  off  applets  and  scripts,  provide  equivalent  information  on  an 
alternative  accessible  page. 

Using  Multimedia  in  a Web  Page 

Provide  an  associated  auditory  description  or  text  transcript  and  provide  a link  to  the  text  transcript 
for  the  important  information  of  a multimedia  presentation. 

Synchronize  captions  or  auditory  descriptions  of  the  visual  track  for  any  time-based  multimedia 
presentation  with  the  presentation.  At  present  there  are  three  formats  or  languages  that  support  the 
ability  to  synchronize  equivalent  alternatives.  These  formats  are  Apple's  QuickTime,  the  W3C's  SMIL 
(Synchronized  Multimedia  Integration  Language)  and  Microsoft’s  SAMI  (Synchronized  Accessible 
Multimedia  Interchange). 

Interactive  content  that  requires  the  surfer  to  press  a key  should  not  be  time-limited  and  animations 
that  use  text  should  show  the  text  long  enough  for  a slow  reader  to  read  it. 

Using  Alternative  Web  Pages 

1 . If  it  is  not  possible  to  create  an  accessible  page,  provide  a link  to  an  alternative  page  that  uses  W3C 
accessibility  technologies,  has  equivalent  information  and  functionality,  and  is  updated  as  often  as 
the  original  page.  Because  it  is  difficult  to  keep  alternative  pages  up  to  date  with  the  full  content  of 
the  original  page,  alternative  pages  should  be  provided  only  after  all  other  pertinent  techniques 
outlined  in  the  WCAG  have  been  attempted.  One  common  way  to  give  surfers  a choice  is  to  use 
the  following  HTML  near  the  top  of  an  opening  page: 

Welcome  to  the  Web  Accessibility  Page! 

<A  HREF="textversionpath/textversionpage">For  a text  version  of  this  site,  follow  this  link.</A> 

2.  Provide  an  alternative  to  web  form  submission  such  as  phone  number,  fax  number,  e-mail  address, 
or  postal  mail  address  to  submit  information.  Even  though  a form  may  be  accessible,  there  maybe 
other  ways  of  filling  it  out  without  using  the  web  that  are  more  convenient  and  less  time- 
consuming  for  the  surfer  with  disabilities. 

Using  Color  and  Backgrounds  in  a Web  Page 

1 . Use  adequate  contrast  between  text  and  background  colors  as  well  as  colors  used  in  graphics.  Dark 
text  against  a light  background  provides  the  most  contrast  for  people  with  low  vision.  Do  not  use 
colorto  convey  information  unless  the  information  is  also  clear  from  the  markup  and/or  content  of 
the  displayed  text. 

2.  Avoid  using  busy  patterns  or  brightly  colored  background  images.  Do  not  tile  an  image  as  a 
background  that  will  distract  from  the  text  or  make  it  difficult  to  distinguish  between  the 
background  and  foreground  elements. 

3.  Make  sure  your  web  pages  can  be  viewed  on  a monochrome  or  grayscale  monitor.  A web  page  that 
can  be  viewed  in  grayscale  or  monochrome  can  also  be  printed  without  loss  of  information. 
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Using  Hyperlinks  in  a Web  Page 


s.  Use  text  for  links  that  make  sense  when  read  out  of  context.  For  example  a link  that  says  ”Click 
Here”  has  no  meaning  out  of  context.  Link  text  should  be  descriptive,  yet  not  too  long,  for  itmay 
cause  difficulty  for  screen-enlarging  software: 


<A  HREF=”access.htmr>Follow  this  link  to  the  Web  Accessibility  Page.</A>, 
displays  like  this  on  a web  page: 


Eollow  this  link  to  the  Weh  Accessibility  Page. 

While  HTML  code  like  this: 

To  go  to  the  Web  Accessibility  Page,  <A  HREF=”access.html”>click  here.</A> 
displays  like  this  on  a web  page: 


To  go  to  the  Web  Accessibility  Page,  click  here. 

b.  Insert  printable,  non-link  characters  between  links  that  are  adjacent,  such  as  an  asterisk  (*)  or  a 
vertical  line  (]).  Visually  impaired  users  and  screen  readers  may  have  difficulty  distinguishing 
between  links  that  are  separated  only  by  a space. 

To  display  text  links  like  this: 


[ Goals  | Components  | Training  Calendar  | Best  Practices  | Resource  Center  | Staff  | 
Newsletter] 


use  HTML  code  like  this: 

[<A  HREF=”goals.htmr>Goals</A> 

| <A  HREF=”syscomp.html”>Components</A> 
j <A  HREF="training.html”>Training  Calendar</A> 
j <A  HREF="bestdocs.htmr>Best  Practices</A> 

| <A  HREF=”rescenter.htmr>Resource  Center</A> 

| <A  HREF=”staff.htmr’>Staff</A> 
j <A  HREF=”newsletter.html”>Newsletter</A>] 


Conclusion 

As  the  use  of  the  Web  is  perceived  to  be  an  effective  tool  for  dissemination  of  research 
findings  or  for  the  provision  of  asynchronous  instruction,  the  issue  of  accessibility  of  web  page 
information  will  become  more  and  more  relevant.  The  W3C  has  embraced  the  issue  of  accessibility 
through  its  Web  Accessibility  Initiative  and  essentially  thrown  open  the  gates  to  the  virtual  kingdom 
for  persons  with  disabilities.  Web  accessibility  is  a design  issue  and  the  benefits  of  developing 
accessibility  features  in  web  pages  easily  offsets  the  additional  time  and  labor  requirements  for 
authoring  accessible  web  pages.  Because  of  the  commitment  of  the  W3C  to  web  accessibility  and  the 
availability  of  resources  and  tools  for  web  authors  to  use  in  the  development  of  accessible  web  pages, 
new  and  experienced  web  authors  have  a compelling  mandate  for  including  accessibility  features  in  the 
design  of  web  pages. 
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Introduction 

This  guide  provides  suggestions  to  faculty  using  online  forum  to  conduct  effective  online 
discussion.  This  guide  was  developed  in  response  to  the  growing  number  of  instructors  at  Ohio 
University  who  are  considering  or  making  the  transition  to  using  an  online  conferencing  tool  for  course 
discussion.  Online  teaching  is  becoming  a part  of  the  educational  culture  with  its  own  unique 
characteristics,  which  need  to  be  considered  in  order  to  be  successful. 

This  instructional  guide  is  based  on  the  experiences  of  some  online  teachers  and  students  at 
Ohio  University.  While  this  guide  is  not  a prescription  for  conducting  perfect  online  discussions,  it 
offers  practical  information  for  instructors  who  want  ideas  and  techniques  to  teach  online  more 
effectively.  It  is  not  a user  guide,  but  it  is  a useful  collection  of  conceptual  issues  and  online 
instructional  tips  organized  into  the  following  four  areas:  pedagogical,  social,  managerial,  and 
technical. 

L Pedagogical  Aspects: 

> Instructor  as  Facilitator 

In  online  teaching  the  role  of  the  teacher  should  be  transformed  from  a transmitter  to  a 
facilitator,  collaborator  or  moderator.  In  this  role  there  is  a partnership  between  the  faculty  and  student 
in  which  they  work  together  to  achieve  the  course  objectives.  The  following  are  some  suggestions: 

The  Role  of  the  Facilitator: 

♦ The  primary  role  of  the  instructor  in  online  teaching  is  to  create  a learning  environment  that 
motivates  the  students  to  construct  meanings  through  interaction  with  each  other  as  well  as  with 
their  instructor.  The  instructor  and  the  student  should  engage  in  an  active  dialog. 

♦ Online  instructors  should  avoid  the  authoritative  style  especially  when  working  with  adult  learners. 

♦ It  is  the  responsibility  of  the  facilitator  to  build  the  understanding  of  conversational  discussion  and 
to  move  students  away  from  the  debate  style,  which  normally  leads  to  flaming.  To  do  that  the 
instructor  needs  to  encourage  students  to  stick  to  the  subject  of  a particular  line  of  discussion. 

What  the  facilitator  should  do: 

♦ In  online  teaching  the  facilitator  is  responsible  for  keeping  discussions  on  track,  contributing 
special  knowledge  and  insights,  weaving  together  various  discussion  threads  and  course 
components,  and  maintaining  group  harmony. 

♦ To  encourage  participation  the  online  instructor  should  introduce  various  options  of  learning  that 
stimulate  learner  participation  and  interaction. 

♦ The  facilitator  should  move  away  from  the  correspondence  model,  in  which  students  send  in  their 
assignments  to  the  instructor  who  then  sends  back  their  feedback,  to  a model  that  would  allow 
dialogue  and  interaction.  The  online  discussion  forum  should  allow  participants’  personalities  to 
come  across  the  medium. 

♦ One  of  the  biggest  mistakes  that  an  online  facilitator  should  avoid  is  to  treat  the  students  as 
children.  In  fact,  in  the  case  of  adult  participants,  some  online  students  may  be  as  knowledgeable 
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as  the  facilitator  in  a specific  subject;  therefore,  the  facilitator  should  utilize  this  situation  by 
asking  such  students  to  share  their  experience  with  their  classmates. 

^ Instructor  Visibility: 

Instructor  visibility  is  a determinant  factor  for  the  success  of  online  discussion.  Therefore, 
every  online  instructor  needs  to  be  aware  of  different  ways  and  means  by  which  they  can  increase  their 
visibility  in  their  classes.  For  example  the  following  are  some  different  type  of  messages  that  online 
instructors  can  send  to  increase  their  visibility. 

■ Content -related  messages  (i.e.  handouts,  discussion  questions,  notes). 

■ Rules  and  Guidelines  (i.e.  grading  procedures) 

■ Technical  tips  (Internet  addresses,  information  about  how  to  send  an  image  as  an  attachment, 
information  of  how  to  paste  text  or  image). 

■ Responses  (answers  to  students  questions,  feedback). 

■ Announcements  (projects  and  assignments  due  dates,  information  about  guest  speakers). 

The  instructor  can  be  visible  by  modeling  a high  level  of  participation,  which  often 
encourages  the  students  to  enhance  their  own  participation.  The  instructor  who  shows  low  visibility  in 
online  discussion  normally  gives  students  an  excuse  to  reduce  their  participation,  or  even  not  to  show 
up.  They  assume  that  their  instructor  will  not  grade  them  down  because  he/she  is  behaving  in  the  same 
way.  Another  reason  that  makes  instructor  visibility  crucial  in  online  discussion  is  to  minimize  the 
sense  of  isolation  that  distant  students  normally  encounter.  Students  feel  secure,  connected  and  as  if 
they  are  working  in  a collaborative  environment  with  other  students  and  with  their  instructor  if  they 
find  a number  of  new  messages  in  the  forum.  Instructor  visibility  provides  distant  students  with  a sense 
of  belonging  to  a body  that  supplements  the  traditional  classroom.  If  for  any  reason  the  instructor  has  to 
be  absent  he/she  should  notify  the  student  as  soon  as  possible  and  look  for  other  ways  of 
communication  since  the  online  discussion  continues  beyond  class  meeting  time.  (See  Figure  FI  for  an 
example). 

Figure  FI 

Hi  Everyone, 

I wanted  to  let  you  know  that  I will  be  out  of  town  Wed  - Sunday.  I 
should  have  email  access  most  of  that  time  and  do  plan  to  check  my 
email  daily  except  Wed  and  Sunday.  I will  likely  not  be  checking  ACT 
until  I return. 

Please  allow  a couple  days  for  me  to  reply  to  your  emails,  though  - 
just  in  case  there  are  technical  problems. 

I will  be  in  my  office  the  rest  of  today  - so  you  can  email  me  until 
5ish  today  without  any  problems. 

I hope  this  doesn't  cause  any  major  problems. 

Chandra 


^ Feedback: 

♦ Timely  feedback  assires  students  that  their  instructor  is  focusing  on  them  and  following  what  they 
are  doing.  On  the  other  hand,  delayed  feedback  leads  students  to  think  that  their  instructor  is  not 
involved  enough  in  their  learning  process.  Therefore,  instructors  should  expect  to  communicate 
regularly  and  frequently  with  students. 

Diplomatic  feedback  is  motivational  and  will  encourage  students  to  be  enthusiastic  and 
confident  that  they  will  succeed  online.  When  the  instructor  wants  to  write  critical  feedback  it  is 
crucial  for  him/her  to  send  a well-worded  email  message  that  preserves  student  dignity  and  prevents 
embarrassment.  Warm  feedback  indicates  that  the  instructor  understands  and  remembers  that  it  is 
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people  who  are  engaging  in  communication  and  not  software.  However,  if  a student  proves  to  be 
disturbing  to  the  online  discussion  he  or  she  should  receive  a positive  but  clear  message  indicating  that 
the  conduct  is  not  appropriate.  Figure  F2  is  an  example  of  an  effective,  critical  and  diplomatic  feedback 
that  conveys  to  students  that  their  instructor  is  interested  in  their  contributions. 

Figure  F2 

Ismail, 

This  is  a very  well  thought  out  response  I really  like  that  you  point  out  the 
importance  of  the  attitudes  - if  the  learners  are  not  ready  or  do  not  see  the 
importance  of  this  training,  that  is  a critical  non- instructional  barrier  to  learning. 

I find  it  interesting  that  you  don't  ask  any  questions  in  the  needs  analysis  to 
determine  if  there  are  other  ways  than  just  inservice  days  to  support  this  effort. 

I do  like  that  you  suggest  really  exploring  the  goal  of  technology  literacy  as  that  may 
have  very  different  definitions  from  different  parties.  (The  principal  may  think 
something  very  different  from  the  teachers  and  the  ID  team  may  have  a completely 
different  idea.) 

Chandra 


^ Online  Materials 

Although  the  online  instructor’s  role  is  basically  to  facilitate,  he/she  also  should  provide 
relevant  materials  to  enhance  the  learning  process.  As  with  the  student  discussion,  the  materials  need 
to  be  focused  on  the  coursework.  To  help  make  material  relevant,  Instructors  should  develop  questions 
and  activities  for  learners  that  relate  to  the  students’  experiences.  After  instructors  have  the 
biographical  information  of  their  students  they  may  be  able  to  include  material  that  would  appeal  to 
the  learning  interest  of  their  students. 

It  should  be  made  clear  whether  posted  material  is  mandatory  or  optional  so  that  students  may 
mange  their  time  and  prepare  for  responses.  Posting  online  material  such  as  Mini-lectures  and 
handouts  requires  careful  editing  since  they  should  enhance  the  face-to-face  lectures  rather  than  offer  a 
replacement.  Also,  all  postings  should  be  checked  for  accuracy  regularly.  Students  will  become 
frustrated  with  the  instructor  who  posts  a website  address  that  is  no  longer  there. 

Be  sure  to  edit  all  online  course  materials  for  correct  information.  All  online  documents 
should  be  clear  and  detailed,  as  well  as  containing  a full  description  of  what  is  required  from  students 
and  the  due  dates  of  these  requirements.  The  instructor  should  have  class  materials  uploaded  and 
tested  2 -3  days  before  the  day  in  which  students  have  to  check  it. 

2.  Social  Aspects: 

Online  education  should  be  looked  upon  as  a social  construct  where  learning  is  supported  by 
technology. 

♦ To  help  build  a sense  of  community  and  reduce  the  feeling  of  isolation,  the  facilitator  should 
encourage  students  to  post  their  biographical  information  in  the  first  online  activity. 

♦ Interpersonal-communication  techniques  are  particularly  important  in  an  online  environment 
where  there  are  fewer  verbal  and  physical  cues  to  help  smooth  dialogue. 

♦ The  energy  and  personality  of  an  online  instructor  can  also  be  effectively  communicated  through 
the  warmth  he/she  brings  to  the  forum.  Many  online  facilitators  believe  that  increasing  warmth 
online  is  a way  to  reduce  the  psychic  distance  among  the  forum  participants. 

285 


799 


♦ Another  way  to  bring  warmth  to  the  forum  is  by  treating  students  uniquely  by  identifying  them  by 
their  names. 

♦ Creating  a friendly,  social  environment  that  promotes  learning  essential  for  the  success  of  an 
online  forum.  Promoting  collaborative  work,  and  providing  the  opportunity  for  other  interactions 
through  the  development  of  a cafe  where  students  are  free  to  discuss  intellectual  and  personal 
matters  on  their  own  can  achieve  this  goal. 

Figure  F3 

Cafe 

Area  for  informal  discussion 

Use  this  space  to  talk  about  whatever  you  want!  Since  you'll  be  seeing  your  team 

members  often,  but  not  others  in  the  class,  this  is  a great  place  to  keep  in  touch. 


3.  Managerial  Aspects: 

Managing  an  online  conference  can  be  time  consuming  and  labor-intensive,  therefore,  instructors  who 

want  to  use  online  discussion  for  the  first  time  should  be  aware  and  prepared  to  accept  the  challenge. 

Once  the  discussion  starts  students  can  create  many  messages  that  could  overwhelm  the  instructor. 

The  following  suggestions  may  help  you  avoid  common  pitfalls. 

^ Organizational: 

♦ Putting  together  an  organized  syllabus  that  establishes  guidelines  and  provides  students  with  clear 
and  detailed  information  can  save  online  instructors  a lot  of  time  and  effort.  The  syllabus  should 
state  in  detail  what  is  expected  when  students  have  to  give  feedback  to  the  class.  It  may  be  useful 
to  provide  both  the  week  and  the  due  date  for  assignments  rather  than  just  writing  “the  feedback  is 
due  on  Monday.” 

♦ Novice  online  instructors  often  underestimate  the  time  and  effort  required  for  planning,  developing 
and  putting  together  online  course  materials.  Many  online  instructors  emphasize  that  lead-time  to 
prepare  an  online  course  is  substantially  greater  than  they  anticipated.  Therefore,  careful  planning 
is  essential. 

♦ A classroom  with  20  students  or  more  will  likely  generate  a huge  number  of  messages  in  a short 
time.  Breaking  the  class  into  small  groups  with  5-7  students  will  ease  the  management  issues. 
Furthermore,  it  will  encourage  more  interactions  among  students  by  keeping  the  number  of 
messages  smaller  and  the  discussion  more  on  track. 

♦ For  organizational  purposes  instructors  may  create  an  area  in  which  they  can  post  announcements, 
contact  information,  office  hours,  and  links  to  related  web  sites.  The  idea  announcement  can  be 

, seen  in  Figure  M4. 
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Figure  M4 
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> Procedural: 

♦ Depending  on  the  objectives  of  the  online  conference  the  instructor  may  decide  to  encourage 
informality  in  discussion.  For  example,  the  instructor  may  decide  to  let  participants  know  that 
perfect  grammar  and  typing  is  less  important  than  making  their  ideas  and  arguments  clear. 

♦ It  is  recommended  that  the  instructor  respond  to  students’  postings  within  72  hours.  The  instructor 
may  not  send  feedback  to  every  student,  however  he/she  can  respond  to  several  students  at  once  by 
weaving  their  contributions  together  or  summarizing  the  discussion. 

♦ The  policy  of  participation  should  be  clearly  stated  in  the  syllabus.  Both  the  importance  of 
participation  to  the  learning  experience,  as  well  as  how  participation  will  contribute  to  the  student 
grade,  should  be  emphasized.  The  instructor  also  needs  to  address  in  the  syllabus  the  issue  of  late 
assignments  and  how  he/she  will  handle  them. 

♦ To  ensure  thoughtful  active  discussions,  students  should  be  graded  on  their  participation  in  them. 
This  grade  should  be  at  least  20-25%  of  the  total  grade  for  the  course. 

^ Administrative: 

♦ When  a discussion  thread  or  a conference  has  served  its  purpose  the  instructor  should  be  decisive 
in  ending  it. 

♦ Students  can  be  given  an  opportunity  to  lead,  moderate,  or  assist  in  directing  the  discussion.  The 
instructor  may  ask  students  to  take  turns  to  experience  administration  responsibility. 

♦ In  any  online  conference  there  are  two  extreme  cases:  an  active  participant  who  appears  above  the 
class  level  and  an  invisible  member  who  contributes  little  to  the  discussion.  The  instructor  needs  to 
privately  contact  these  members  to  ask  the  first  to  wait  a few  responses  before  contributing  (see 
example  M 1 ) and  to  ask  the  second  to  participate  more  actively  (see  example  M2). 


Ml 

Overachiever 

Mary,  I like  your  thoughtful  and  critical  reflections 
for  the  discussion  topics.  My  only  concern  that  your 
classmates  are  left  behind  and  they  have  little  to  add 
to  the  discussion.  I’d  appreciate  if  you’d  slow  down 
a little  bit  so  that  your  classmates  can  give  it  a shot. 
Thanks 


M2 

Lurker 

John,  I am  a little  bit  worried  about  your 
contribution  to  the  discussion  forum.  You  have  only 
posted  one  message  in  the  last  two  weeks.  Is  there 
any  problem  that  is  holding  you  back  from  the  class 
discussion.  You  need  to  move  forward  and  to  catch 
up  with  the  group.  Just  remember  that  25%  of  your 
grade  comes  from  your  participation  in  the 
discussion.  Thanks. 
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♦ Conflict  may  erupt  in  online  discussion,  especially  when  the  topic  of  the  discussion  is  somewhat 
controversial.  In  many  cases  that  can  lead  to  “flaming”.  These  remarks  are  not  productive  and  may 
destroy  the  discussion.  To  guard  against  conflicts  instructors  need  to  set  up  norms  and  rules  for  the 
online  conference  that  lay  the  foundation  for  a conflict  management  strategy.  For  example,  the 
instructor  can  set  a rule  at  the  beginning  of  the  conference  to  asking  students  to  respect  all  opinions 
and  not  impose  their  views  on  others. 

4.  Technical  Aspects: 

In  an  online  environment,  problems  related  to  hardware  and  software  will  likely  arise.  So, 
while  remaining  focused  on  the  learning  process,  the  online  instructor  must  develop  a working 
understanding  of  the  technology  s/he  is  using. 

♦ The  instructor  should  provide  students  with  a user  guide  or  website  for  the  online  conference  that 
addresses  both  the  content  and  the  common  technical  problems.  Novice  users  need  time  so  that 
they  can  be  comfortable  with  the  technology.  Therefore,  the  instructor  should  give  students 
adequate  time  to  learn  the  system  before  they  participate 

♦ Before  starting  an  online  conference  instructors  need  to  acquire  the  appropriate  training  in 
configuring,  manipulating,  and  managing  the  conference.  A step-by-step,  face-to-face  tutorial  for 
novice  computer  conference  administrator  is  recommended.  The  instructor  should  be  aware  of  who 
is  available  for  technical  support  beyond  his/her  technical  skill.  Contact  information  about  the 
technical  support  personnel  should  be  made  available  to  students  in  the  syllabus. 

♦ Using  a graduate  or  teaching  assistant  who  has  technical  skill  is  important  for  the  success  of  the 
conference.  Graduate  or  teaching  assistants  can  help  with  technical  issues,  as  well  as  sharing 
administrative  and  teaching  load.  For  example,  a teaching  assistant  can  reply  to  many  technical 
questions  or  send  feedback  related  to  the  discussion,  or  even  step  in  to  direct  the  discussion  in  the 
instance  of  the  instructor’s  absence. 

For  more  information 

Bonk,  C.J.  and  King,  K.S.  (1998).  Electronic  collaborations:  Learner-centered  technologies 
for  literacy,  apprenticeship,  and  discourse.  Mahwah;  NJ:  Lawrence  Erlbaum  Associates. 

Koschmann,  T.  (1996).  CSCL:  Theory  and  Practice  of  an  emerging  paradigm  (computers, 
cognition,  and  work).  Mahwah;  NJ:  Lawrence  Erlbaum  Associates. 

Harasim,  L.,  Hiltz,  S.R.  Teles,  L.,  Turoff,  M.  (1995).  Learning  networks:  A field  guide  to 
teaching  and  learning  online.  Cambridge,  MA;  MIT  Press. 

Phipps,  R.A.  ; Wellman,  J.V.  , Merisotis,  J.  P.  (1998).  Assuring  quality  in  distance  learning:  A 
preliminary  review.  Washington,  DC:  Council  for  Higher  Education  Association. 

Hillesheim,  G.  (1998).  Distance  learning:  Barriers  and  strategies  for  students  and  faculty.  The 
Internet  and  Higher  Education,  1 (1 ),  3 1 -44. 
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Contextual  considerations  for  teaching  online  at  the 

POSTSECONDARY  LEVEL 


Salley  Sawyer 
University  of  Nevada 
Joi  Moore 

U n i vers  i ty  of  Missouri 

Designing  an  online  class  guided  by  principals  of  learner-centered  instruction  at  the 
postsecondary  level  is  a daunting  challenge  for  new  professors  facing  the  demands  of  a tenure 
track  position.  The  case  study  identified  challenges  faced  by  a new  professor  teaching  an 
online  course.  This  study  found  an  online  environment  presents  challenges  unique  to  the 
technology  and  recommends  that  university  administrators  must  recognize  such  challenges  in 
order  to  encourage  pedagogically  sound  online  instruction. 

The  Study’s  Purpose 


This  paper  presents  the  findings  of  a pilot  project  on  two  junior  faculty’s  experiences  with 
online  course  development  and  teaching  in  a university  setting.  The  study’s  purpose  was  to  identify 
contextual  issues  faced  by  new,  untenured  university  faculty.  Often  junior  professors  are  learning  to 
teach  in  the  face-to-face  environment  as  well  as  learning  to  teach  in  the  relatively  new  online 
environment.  Additionally,  the  current  emphasis  on  integrating  technology  into  postsecondary 
teaching  practices  as  evidenced  by  the  Pew  Symposia  in  Learning  and  Technology 
(http.VAyww, center.rpi.edu/PewSviri  1 .htniH  and  the  academic  reward  system  as  evidenced  by  the  tenure 
process  can  mean  that  new  faculty  often  encounter  overlapping  and  or  competing  demands  on  their 
time  and  efforts  as  they  seek  success  in  meeting  these  two  challenges.  This  study  was  designed  to 
identify  and  describe  the  contextual  experiences  of  junior  faculty  as  they  developed  and  taught  online 
courses.  The  study’s  purpose  was  to  identify  and  to  describe  those  experiences  in  ways  that  would  be 
useful  to  academic  administrators,  and  in  ways  that  would  provide  a robust  basis  for  conducting  a 
larger  descriptive  study. 

The  researchers  designed  a qualitative  case  study  examining  two  professors  who  were  in  the 
mid  stages  of  the  tenure  process,  who  had  experience  teaching  courses  online  and  face-to-face,  and 
who  facilitated  active,  meaningful  learning  that  can  be  termed  leamer-centered  instruction  according  to 
Branch  ( 1 995)  and  McCombs  and  Whisler  ( 1 997).  The  researchers’  primary  purposes  were: 

• To  clarify  contextual  issues  that  affect  a professor’s  ability  to  successfully  integrate  online 
teaching  technologies  into  instructional  practices  that  support  active  and  meaningful  learning 

• To  describe  the  professors’  perceptions  and  experiences  in  online  teaching 
The  researchers’  secondary  purposes  were: 

• To  identify  any  additional  data  sources  that  would  provide  a fuller  description  of  this  study’s 
area  of  interest 

• To  develop  useful  questions  in  order  to  construct  a larger  study  that  will  continue  to  further 
define  critical  issues 

Research  Method 
Qualitative  Case  Study 

Participant  Selection 

The  research  design  selected  for  this  study  is  a qualitative  case  study  following  the  guidelines  for  a 
case  study  as  explained  by  Merriam  (1998).  The  case  is  bounded  by  the  area  of  interest,  the  focus  of  the 
research  questions,  and  by  the  selection  of  the  participants.  Based  upon  LeCompte  and  Preissle’s 
(1993)  recommendations  for  participant  selection,  selection  criteria  were  established.  The  two 
participants  selected  for  the  study  met  the  following  qualities,  they: 

• Valued  and  followed  good  teaching  practices  as  described  by  Chickering  and  Gamson  (1987) 
and  McCombs  and  Whisler  (1997). 

• Were  in  a tenure  track  position  and  were  in  the  process  of  going  for  tenure 
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• Were  developing  their  experience  in  teaching  with  online  technologies 

• Had  a personal  interest  in  becoming  better  at  teaching  with  online  technologies 

• Were  teaching  at  two  large  universities  that  have  officially  adopted  a course  management 

software 

• Were  teaching  an  online  class  in  two  different  disciplines 

• Represented  the  use  of  two  different  brands  of  online  course  software 

The  two  participants  agreed  to  participate  in  the  pilot  study  because  they  had  a personal 
interest  in  the  topic.  When  the  data  were  collected  and  transcribed,  the  researchers  assigned 
pseudonyms  to  the  two  participants.  The  transcribed  interviews  were  returned  to  the  participants.  The 
participants  have  not  yet  seen  the  completed  analysis. 

Data  Collection 

One  tape-recorded  interview  was  conducted  with  each  participant.  The  interviews  were  open 
ended  and  guided  by  the  suggestions  of  Kvale  (1996).  The  researcher  let  the  interview  process  guide 
the  questions  rather  than  following  a question  guide.  Each  interview  lasted  until  the  interviewer  and  the 
participant  sensed  that  the  discussion  and  questions  were  exhausted.  Each  interview  lasted 
approximately  one  hour.  The  interviews  were  fully  transcribed  and  the  analysis  made  from  the 
transcriptions. 

Data  Analysis 

The  first  author  read  through  the  transcribed  interviews,  took  notes  in  the  margin  and 
underlined  sentences  or  phrases  that  appeared  to  describe  each  participant’s  experiences.  This  process 
resembled  the  coarse  coding  described  by  Merriam  ( 1 998).  During  a second  reading  of  the  transcribed 
data,  the  first  author  began  constructing  three  charts  that  displayed  and  summarized  the  categories  of 
data.  One  chart  (Figure  A)  displayed  information  describing  the  participants  in  their  setting.  The 
second  chart  (Figure  B)  displayed  the  similarities  between  the  two  participants.  The  third  chart  (Figure 
C)  displayed  the  differences  between  each  of  the  two  participants’  experiences. 

As  the  first  author  created  the  charts,  she  also  began  to  create  a list  of  categories  pulled  from 
the  transcriptions.  This  list  was  kept  on  a lined  notepad  beside  the  transcribed  pages.  This  list  of 
categories  became  the  basis  for  analyzing,  refining,  and  representing  the  data  in  ways  that  would  meet 
the  purpose  of  the  study  and  answer  the  research  questions.  These  categories  were  then  re-analyzed 
and  combined  according  to  their  supporting  data  units. 

Self-disclosure  of  the  researchers 

The  two  researchers  were  drawn  to  designing  a study  to  explore  this  issue  because  they  have  had 
personal  experience  with  creating  and  teaching  online  courses  for  university  students.  Their  personal 
experience  facilitated  their  perception  that  a professor’s  experiences  when  designing  and  teaching  an 
online  course  were  different  than  the  experiences  when  designing  and  teaching  a similar  face-to-face 
class.  Because  the  two  researchers  are  not  tenured  faculty,  they  became  aware  that  the  demands  of  the 
tenure  process  seem  neither  to  reflect  nor  to  recognize  the  differences  between  teaching  in  a face-to- 
face  class  and  teaching  in  an  online  class.  Thus,  the  two  researchers  had  a personal  interest  in 
developing  a research  study  that  would  identify  and  describe  the  contextual  considerations  new 
professors  face  when  teaching  an  online  course.  The  two  researchers  knew  one  another  from  being 
classmates  in  a Ph.D.  program  at  the  Uni  versity  of  Georgia. 

The  Research  Questions 

• How  do  the  professors  describe  the  experience  of  planning  and  teaching  an  online  course? 

• How  do  the  professors  perceive  the  demands  of  teaching  a class  online  compare  with  the 
demands  of  teaching  a similar  class  face-to-face? 

• How  do  the  professors  understand  the  context  of  online  course  development  and  teaching? 

Representing  the  Data 

The  two  professors 

The  two  professors  in  this  study,  Professor  Eden  and  Professor  Simpson,  teach  at  major 
universities  in  the  United  States.  Professor  Eden  is  teaching  in  a college  of  education  at  a Carnegie 
ranked  Research  I University;  Professor  Simpson  is  teaching  in  a college  of  hotel  administration  at  a 
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doctoral  granting  university.  Both  professors  are  teaching  required  courses  in  their  respective  colleges. 
They  are  each  using  a well-known  brand  of  online  course  software.  During  the  period  of  this  study, 
both  professors  were  teaching  online  courses  for  the  second  time,  and  both  were  confronting  the 
demands  of  the  tenure  process. 

Both  professors  were  technologically  fluent  in  that  they  had  created  web  pages  for  their  courses 
prior  to  using  the  online  course  software,  and  both  had  originally  started  making  pages  by  hand  coding 
with  HTML.  Both  professors  valued  active  learning  for  their  students  and  sought  to  use  strategies  that 
would  facilitate  meaningful  knowledge  construction.  Although  the  professors  did  not  use  the  term 
learner-centered  to  describe  their  approach  to  teaching  and  learning,  they  both  described  their  concerns 
and  teaching  strategies  in  ways  that  meshed  with  Branch  (1995)  and  McCombs  and  Whisler’s  (1997) 
description  of  learner-centered. 


Professor 

University 

College 

Course 

Class 

Size 

Students 

Software 

Eden 

Primarily 
residential, 
Research  Level 

I 

College  of 
Education 

Educational 

Technology 

Required 

15 

Graduate 

In-service 

Teachers 

Blackboard 

Simpson 

Urban, 

commuter 

campus, 

Doctoral 

granting 

College  of  Hotel 
Administration 

Human 

Resources 

Required 

12 

Undergraduate 

Working  in 
hospitality 

WebCT 

Figure  A.  Chart  displaying  information  describing  the  participants  in  their  setting 


The  students  in  both  classes  were  working  in  fields  related  to  the  course.  Professor  Eden’s 
students  were  predominantly  K- 1 2 school  teachers.  Many  were  enrolled  in  the  masters  program 
because  they  wanted  to  advance  to  a higher  pay  level.  Professor  Simpson  required  the  students  to  be 
working  at  least  part  time  in  the  hospitality  industry  as  a pre-requisite  for  enrolling  in  the  class,  and 
gave  the  students  assignments  that  required  the  students  to  interact  with  the  human  resources  personnel 
in  that  industry.  The  students  in  both  professors’  courses  could  choose  to  take  the  course  online  or  in  a 
face-to-face  classroom  environment. 

Both  professors  wanted  to  be  better  at  teaching.  Professor  Simpson  stated, 

To  get  tenure  you  need  to  be  able  to  teach  at  a standard  that’s  acceptable  to  the  university,  and  that 
standard  is  not  acceptable  to  me.  I needed  to  push  myself  to  learn  how  to  teach.  So,  I took  a 
certification  class,  it’s  called  a certified  hospitality  educator,  which  was  a week  long  class. 


Professor 

Technology 

fluency 

Online 

software 

Teaching 

goals 

Importance 
of  feedback 

Course 

development 

Eden 

Created  their 
own  resource 
rich  web  pages 
when  teaching 
a prior  class. 

Had  to  spend 
time  on  their 
own  to  learn  to 
use  the 
university 
online 
courseware 

Wanted  to  be 
better  teachers 
than  what  their 
administrators 
suggested  was 
acceptable 

Believed 
students  need 
good 

feedback  in 
order  to 
learn. 

Learned  about 
teaching  online 
from  the 
students’ 
feedback, 
interactions  and 
assignments. 

Simpson 

Figure  B,  Chart  displaying  similarities  between  the  two  participants 

Both  professors  believed  that  their  online  courses  had  to  be  more  than  just  reading  and  completing 
the  assignments.  Professor  Eden  explained, 

How  are  you  really  going  to  get  them  to  think?  ....  I mean  if  all  I had  to  do  was  pick  up  a 
book  and  read  materials,  then  I guess  universities  are  not  really  necessary  ....  So,  I mean  just 
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having  a web  site  with  your  information  on  there,  and  having  those  discussion  boards  is  not 
doing  it.  Because  the  discussion  boards  are  hard  to  manage,  people  are  not  reading 
everything,  and  you’re  not  really  getting  the  interaction  between  the  students  that  you  want 

Neither  Professor  Eden  nor  Professors  Simpson  had  definitive  answers  to  the  challenges  they  were 
encountering  as  the  developed  their  online  courses.  They  were  both  actively  learning  as  they  went 
along.  Both  professors  were  learning  from  their  students. 

The  two  professors  differed  in  some  of  the  online  software  usage  and  in  the  ways  they  were 
encouraged  to  teach  an  online  course.  Professor  Simpson  was  given  release  time  from  the  department 
chair  to  develop  the  course.  The  professor  considered  online  learning  to  be  of  growing  importance  to 
the  hospitality  field.  The  department  let  the  class  “go”  even  though  its  enrollment  of  12  was  less  than 
the  required  number  for  running  a course. 

Professor  Eden  felt  somewhat  pushed  into  teaching  an  online  course  because  the  administration 
wanted  to  get  more  of  the  curriculum  online.  She  was  worried  that  the  course  would  not  be  appropriate 
for  online  learning.  “I  didn’t  think  the  class  was  appropriate  because  it  was  team  based,  and  I don’t 
care  if  you  do  e-mail,  you  still  want  to  meet  face-to-face  in  a team  kind  of  thing.” 

The  two  professor  experienced  differences  in  the  way  that  their  universities  marketed  the  online 
classes.  Professor  Eden’s  university  promotes  online  courses  and  online  programs  on  the  web  site  by 
appealing  to  people  who  don’t  have  sufficient  time  to  take  a class.  Professor  Eden  explained, 

The  way  that  it  markets  is  probably  the  reason  why  we  have  the  attitude  problem  of  our  students. 
The  web  site  says,  ‘You  don’t  have  time  to  take  a class?  You  don’t  have  time  for  class?’  So,  what  does 
that  mean?  ....  Which  means  that  you  don’t  have  time  to  do  all  the  work  that  goes  with  the  class,  then 
take  web  based  classes. 
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with 

individual 
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Students 
did  not  report 
interface 
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Figure  B.  Chart  displaying  differences  between  the  two  participants  * experiences 


In  contrast,  Professor  Simpson’s  university  does  very  little  marketing  of  the  online  courses  and  has 
no  online  programs.  Professor  Simpson  attributes  some  of  the  class’s  low  enrollment  to  a lack  of 
marketing.  “I  don’t  know  what  distance  ed  said  to  them  over  there.  The  two  flyers  that  I saw  were  not 
acceptable  to  me  to  find  out  information,  and  so  I will  be  creating  my  own  marketing  this  year  for  fall 
semester.” 

Because  Professor  Eden’s  students  were  taking  courses  at  the  university  that  were  presented  with 
different  course  software,  and  because  part  of  their  course  included  designing  websites,  they  expressed 
their  confusion  about  finding  how  professors’  organized  their  online  courses.  “People  organize 
information  differently.  So,  I might  put  my  assignment  in  Assignments.  Some  people  put  all  their 

assignments  in  Course  Documents Where  do  the  students  look  for  just  this  document  to  help  them 

do  something?” 

Summary 

Although  the  professors  had  many  differences  between  their  institutions  and  their  fields  of  study, 
both  professors  shared  a strong  belief  in  active  learning  and  leamer-centered  instruction.  Both 
professors  were  motivated  by  a desire  to  create  a positive  and  meaningful  learning  experience  for  their 
students.  Both  professors  needed  interaction  and  feedback  from  their  students  in  order  to  gauge  the 
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progress  of  their  students’  knowledge  development.  Both  professors  created  projects  or  assignments  in 
their  online  class  to  help  their  students  learn  and  to  enable  the  professors  to  understand  how  the 
students  were  progressing. 

Findings  in  Response  to  the  Research  Questions 


How  do  the  professors  describe  the  experience  of  planning  and  teaching  an  online  course? 

The  two  professors  found  their  time  management  and  preparation  skills  differ  when  they  teach  in 
an  online  environment.  Both  professors  were  learning  to  develop  their  online  teaching  and  course 
development  skills.  The  two  professors  discussed  changes  in  their  thinking  as  a result  of  developing 
and  teaching  online.  Professor  Simpson  reported  that  the  experience  of  both  taking  an  online  course 
and  creating  an  online  course  caused  her  to  think  differently  about  learning.  She  explained,  “I’ll  talk 
with  my  students  about  where  do  they  leam  best.  And  if  they  say,  ‘I  leam  the  best  curled  up  on  my 
couch.’  OK.  Can  you  get  you  computer  there?” 

For  Professor  Eden,  the  online  course  caused  her  to  rethink  her  frequency  of  online  responses  as  it 
related  to  the  students’  ability  to  manage  themselves.. 

The  first  time  I taught,  I tried  to  respond  to  every  other  message  ....  I started  letting  them  control 
as  far  as  like  responding  to  each  other.  And  I would  add  something  in  if  I felt  they  were  off  target  or 
getting  out  of  hand,  you  know  a little  bickering  and  so  forth  ....  But  I felt  I was  taking  up  too  much 
time  trying  to  make  sure  they  were  doing  what  they  needed  to  do. 


How  do  the  professors  perceive  the  demands  of  teaching  a class  online  compare  with  the  demands  of 
teaching  a similar  class  face-to-face? 

Both  professors  remarked  about  the  different  time  expenditure  between  an  online  course  and  a 
face-to-face  class.  Professor  Eden  stated, 

It’s  much  more  work  than  face-to-face.  You  definitely  have  to  be  more  prepared,  far  more  in 
advance.  A lot  of  time  when  I was  doing  face-to-face  I’d  think,  I want  to  change  that  a little  bit  because 
of  the  way  they  reacted  maybe  the  week  before  and  I could  prepare  the  night  before. 

Professor  Simpson  was  even  more  specific  in  the  time  differences. 

I logged  an  average  of  six  hours  a week  in  WebCT  in  my  class  which  is  more  than  I ever  put  in  on 
class  time.  ...  So,  maybe  four  and  a half  hours  max  a week  in  a face-to-face  class  that  I may  teach  two 
sections  of  versus  six  hours  a week  in  an  online  course.  And  that’s  not  counting  the  grading  and 
inputting.  I knew  what  it  took  to  grade  a project  that  I taught  in  a face-to-face  class.  That  same  project 
or  some  elements  of  the  project  to  grade  in  an  online  class,  write  up  the  results,  post  the  results,  do 
follow-up,  adds  twice  the  amount  of  time. 

How  do  the  professors  understand  the  context  of  online  course  development  and  teaching? 

Both  Professor  Eden  and  Professors  Simpson  felt  that  the  university  administration  did  not 
have  a realistic  understanding  of  the  time  demands  and  demands  for  self-teaching  that  an  online  course 
required.  Both  professors  had  to  spend  time  and  energy  figuring  out  the  technology  and  the  pedagogy 
of  online  teaching  for  themselves. 

Both  professors  were  strongly  encouraged  by  their  college  administrators  to  use  online  course 
software  although  the  quality  of  their  support  varied.  Both  professors  knew  that  university 
administrators  anticipated  being  able  to  teach  larger  classes  with  online  technologies,  and  both 
professors  expressed  concern  for  the  quality  and  course  management  of  classes  with  larger  enrollments 
than  25  or  30  students. 

Teaching  an  online  course  forced  the  professors  to  confront  the  technology  and  the  software. 
Professor  Simpson  designed  elements  into  the  course,  such  as  audio  clips  or  animated  gifs,  in  order  to 
check  capability  of  the  students’  computers.  “Throughout  the  course,  I kept  checking  with  the  students 
to  make  sure  they  still  had  computers  at  home.  And  two  students  either  lost  their  computers  because 
they  moved  and  it  was  the  room  mate’s  computer,  or  the  computer  failed  on  them.” 
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Discussion  of  the  Findings 

Interaction 

In  the  context  of  teaching  their  online  courses,  interaction  became  an  important  concept. 
Interaction  infused  both  professors’  thinking.  The  professors  sought  to  create  interactions  on  many 
levels.  The  interaction  between  the  professors  and  the  students  and  the  course  content  and  the  students’ 
work  experiences  and  the  technology  was  complex  and  was  an  important  part  of  the  professors’ 
thinking  about  their  online  courses.  The  professors  encouraged  interaction  between  the  students  and 
the  course  content.  The  students  in  both  professors  classes  were  working  in  the  field  of  study,  and  the 
two  professors  deliberately  structured  their  courses  in  ways  that  would  make  the  course  content  interact 
with  the  students’  work  life  experiences. 

The  professor  interacted  with  the  students  and  learned  from  their  students’  interactions. 
Through  the  students’  interaction,  the  professors  learned  how  well  the  students  were  progressing  in 
their  knowledge  development  as  well  as  how  effective  the  technology  and  course  organization  was.  In 
sum,  the  professors  learned  about  how  to  create,  organize,  and  teach  their  online  courses  from 
interactions  with  their  students. 

The  Importance  of  the  Students  Expectations 

The  two  professors’  online  teaching  experiences  were  very  influenced  by  the  students’ 
expectations.  The  students’  expectations,  as  described  by  the  two  professors,  were  related  to  their 
reasons  for  taking  the  courses  and  the  university’s  conceptualization  of  online  learning  as  manifested 
through  their  outreach  marketing.  Professors  Eden’s  students  were  primarily  working  K-12  teachers 
whose  motivation  for  earning  a graduate  degree  was  to  move  into  a higher  pay  grade  level  in  their 
school  systems.  The  university  marketed  the  online  learning  program  by  implying  that  online  learning 
was  easier  and  required  less  time  than  coming  to  the  campus  to  attend  classes.  The  university’s 
marketing  efforts  influenced  the  students’  expectations  that  the  course  would  make  it  easier  for  them  to 
manage  their  time. 

Tenure  considerations 

There  were  differences  between  the  pressures  the  two  professors  experienced  in  regard  to 
tenure  documentation.  Professor  Simpson  considered  her  field  more  applied  than  theoretical. 
Although  there  was  pressure  to  achieve  excellence  in  at  least  one  of  the  categories,  achieving 
excellence  in  the  teaching  category  was  sufficient  to  carry  the  lack  of  an  excellence  rating  in  the 
research  category.  However,  this  put  even  more  pressure  on  ensuring  a good  rapport  and  evaluation 
from  the  students  in  the  online  course.  Professor  Eden  faced  greater  pressure  to  publish  and  conduct 
research. 

Implications  and  Recommendations 

Implications  for  university  administrators  based  upon  the  study 

• Universities  must  provide  more  specific  training  support  for  professors  using  online 
technologies. 

• Online  courses  are  not  a time  saver  for  faculty. 

• Universities  must  be  critical  about  the  way  online  courses  are  marketed  to  potential  students 

Recommendations  for  further  study 

The  findings  in  this  pilot  study  have  served  to  illuminate  the  areas  that  need  further  data  collection 
in  order  to  design  a more  robust  study. 

• Further  interviews  and  data  needs  to  be  collected  to  explain  why  the  university  administration 
is  urging  new  professors  to  teach  courses  using  online  technologies. 

• Further  interviews  and  data  need  to  be  collected  from  more  professors  in  order  to  exhaust  all 
the  questions  that  were  raised  in  this  pilot  study 

• A fuller  search  of  the  literature  related  to  the  context  of  online  teaching  needs  to  be  developed. 
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A COMPETITION  TO  PROMOTE  THE  DESIGN  OF  EDUCATIONAL  SOFTWARE 


Brad  Hokanson 
Simon  Hooper 
Paul  Bernhardt 
University  of  Minnesota 


We  established  the  University  of  Minnesota  Learning  Software  Awards  as  a mechanism  to  identify  and 
promote  exemplary  and  innovative  educational  software.  Our  long-term  goal  is  to  establish  an 
electronic  forum  featuring  best  practices  in  the  field.  We  want  to  influence  educational  design  by 
providing  access  to  samples  of  work  judged  to  be  exceptional,  and  to  create  a form  of  eJoumal  that 
publishes  active  samples  of  exemplary  practice-in-action,  not  simply  written  articles. 

One  criticism  of  our  field  is  that  we  have  not  identified  what  we  consider  to  be  exemplary  projects. 
Although  many  design  professions  have  well  established  and  easily  accessible  archives  of  their  best 
work  (e.g.  art  galleries,  buildings,  and  books),  the  field  of  Instructional  Technology  has  done  little  to 
establish  resources  where  students  or  practicing  designers  can  examine  the  best  and  most  inspirational 
works. 

During  the  fall  of  1999  and  spring  of  2000,  we  solicited  and  received  approximately  65  entries  from 
across  the  United  States.  Judging  involved  a two-phase  process.  First,  a pool  of  35  judges  screened 
entries.  Twenty  entries  were  forwarded  for  further  consideration.  Three  expert  judges  conducted  final 
judging:  Tom  Duffy  (UNext),  Lloyd  Rieber  (University  of  Georgia),  and  Stan  Troll  ip  (Capella 
University).  The  judges  selected  the  following  award  winners  and  honorable  mentions. 

First  place  awards 

The  Writing  Trek  by  Sunburst  Communications 
Math  Problem  Solving  by  Plato  Education 

Psychology  Experiments  by  Kenneth  McGraw,  University  of  Mississippi 
Off  the  Wall  by  Michael  Gardner,  University  of  Georgia 

Runners-up 

Research  Assistant  by  Dan  Schuch,  Florida  State  University 
Tuberculosis  Case  Management  by  University  Research  Co. 

Honorable  mentions 

Claymation  by  Mary  Beth  Kiser,  Edina  Public  Schools,  MN 
Probability  Explorer  by  Hollylynne  Stohl  Drier,  University  of  Virginia 
Bohr’s  Atomic  Model  by  Jeff  Wilden,  Weber  State  University 

Although  it  was  beyond  the  scope  of  the  competition  to  provide  full-access  to  the  projects,  all  the 
winners  provided  resources  that  can  be  accessed  and  examined.  Samples  representing  their  projects  are 
available  at  the  competition  website:  http://design.urnn.edu/IearningcompetitionIn  addition,  each  of  the 
winners  has  submitted  an  article  for  a special  issue  of  Tech  Trends  outlining  the  problems  they 
addressed  in  their  projects  and  commenting  on  how  they  designed  their  solutions. 

Our  primary  goal  was  to  support  the  development  of  innovative  uses  of  technology.  We  focused  on 
three  dimensions  of  innovation  for  this  competition.  The  first  dimension  involves  using  new 
capabilities  of  technology  for  educational  advantage.  For  example,  the  very  scale  and  accessibility  of 
the  Internet  is  used  to  the  advantage  of  researchers  and  students  alike  through  Psychology  Experiments 
by  Kenneth  McGraw.  There,  the  power  of  the  Internet  connects  researchers  and  research  subjects,  and 
provides  educational  examples  for  study.  Off  the  Wall  by  Michael  Gardner  uses  the  interactive 
capabilities  of  multimedia  to  investigate  the  artist  Chuck  Close.  Users  explore  the  work  of  the  artist, 
interacting  with  elements  of  scale,  brush  stroke  and  subject.  Jeff  Wilden  describes  an  interactive 
multimedia  program  titled  Bohr’s  Atomic  Model.  This  web-based  interactive  simulation  allows 
students  to  build  an  atom  using  an  atomic  construction  set. 
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The  second  dimension  involves  solving  existing  problems  in  a new  or  unusual  way.  Plato's  Math 
Problem  Solver  and  Sunburst's  Writing  Trek  use  unique  capabilities  of  the  computer  and  the  Internet  to 
assist  learners  while  providing  diverse  and  challenging  learning  environments.  Plato’s  Math  Problem 
Solving  software  uses  real-world  math  problems  and  employs  three  levels  of  scaffolded  support  to 
guide  the  user  through  the  solution  process.  Writing  Trek  includes  a great  variety  of  writing  activities 
and  provides  the  opportunity  to  publish  student  work  on  an  Internet  site.  In  contrast  to  these 
commercially  developed  products,  Dan  Schuch  created  Research  Assistant,  a database,  to  support  the 
complex  cognitive  demands  of  graduate  education.  Not  all  winners  were  fully  computer-based.  Mary 
Beth  Kiser’s  fifth  grade  class  completed  a learning  activity  involving  clay,  video,  and  the  World  Wide 
Web. 

The  third  dimension  includes  new,  under-addressed,  or  under-served  problems.  Tuberculosis  Case 
Management  by  Elisa  Knebel  and  Probability  Explorer  by  Hollylynne  Drier  use  the  capabilities  of  the 
computer  in  this  manner.  Tuberculosis  Case  Management  helps  medical  workers  in  the  Third  World, 
far  from  Silicon  Valley,  diagnose  and  treat  a deadly  disease.  Probability  Explorer  is  oriented  to  younger 
students.  It  engages  children  though  probability  experiments  and  exploration,  and  allows  them  to 
construct  a more  accurate  understanding  of  the  nature  of  chance. 

Our  observations  from  the  competition  provide  insight  into  the  development  of  educational 
software.  First,  it  is  clear  from  the  descriptions  presented  by  the  award  winners  that  the  design 
processes  employed  varied  greatly  across  the  projects.  Some  groups  followed  formal  methodologies, 
and  in  some  cases,  the  strategies  employed  are  the  result  of  a formal  design  process.  In  other  cases, 
however,  the  innovation  appears  centered  on  the  overlap  between  expertise  in  the  subject  matter  and  a 
use  or  interest  in  a technology.  Often,  creative  inspiration  appears  to  occur  as  sudden  insight. 

Second,  access  to  development  funds  did  not  limit  creativity.  As  noted  above,  three  winners  were 
graduate  students.  Apparently,  innovation  does  not  require  large  teams  or  budgets.  Third,  comments 
from  the  competition  judges  about  the  value  of  the  judging  process  were  overwhelmingly  positive. 
Many  people  commented  that  the  process  of  reviewing  competition  entries  was  enlightening  and 
worthwhile. 


The  competition  was  funded  by  the  Design  Institute  at  the  University  of  Minnesota.  The  second  round 
of  the  competition  is  currently  beingplanned.  Information  about  the  competition , including  a video 
and  samples  of  this  year’s  winning  entries,  can  be  accessed  at  the  competition  website. 
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The  Paradox  of  Constructivist  Instruction:  A Communicative 
Constructivist  Perspective 

Rocci  Luppicini 
Concordia  University 


Abstract 

With  the  advent  of  Constructivist  oriented  instruction  in  learning  institutions  comes  an 
enormous  challenge,  that  of  structuring  individual-centered  learning  within  a community  of 
learners.  Within  this  instructional  framework,  struggles  to  satisfy  both  individual  and  group 
learning  needs  can  lead  to  one  canceling  the  other  out.  Tendencies  towards  either  individual- 
centered  or  collaborative  learning  depend  largely  on  the  Constructivist  stance  adopted 
(Cunningham,  1 991,  Garrison,  1998;  Jonassen,  1991,  1995;  Lave  & Wenger,  1991;  Vygotsky, 
1987).  This  ongoing  struggle  represents  the  paradox  of  Constructivist  instruction;  however,  there 
is  an  alternative.  This  exploratory  paper  tilizes  assumption  from  Internal  Realism  (Putnum,  1991, 

1 994)  draws  from  advances  in  cybernetic  science  (Maturana  & Varela,  1 980;  Varela,  1981)  and 
communications  theory  (Habermas,  1990;  Krippendorff,l  994)  to  make  the  case  that  underlying 
biological  and  communicative  structuring  play  an  important  constitutive  role  in  multiple  levels  of 
meaning  construction  (biological,  psychological,  social)  that  are  implicated  in  learning  processes. 
The  extent  to  which  knowledge  of  underlying  structures  can  inform  on  learning  processes  is 
addressed  and  recommendations  are  made  for  adopting  Communicative  Constructivist 
Perspective  (CCP)  as  a potential  educational  reform  tool  for  increasing  awareness  of  instruction 
that  may  detract  from  efforts  to  achieve  sustainable  learning  within  a community  of  learners. 


I.  Assumption 

This  exploratory  paper  revolves  around  a conceptualization  of  the  subject-object  distinction  held 
by  a variation  of  realist  philosophy  that  emphasizes  the  importance  of  human  practices.  This  position  is 
commonly  referred  to  as  ’internal  realism.'  Internal  realists  take  into  consideration  scientific  and  everyday 
practices  while  utilizing  subject-object  conceptual  distinctions  to  deal  with  conflicting  knowledge  claims 
in  order  to  achieve  rational  consensus  in  a changing  world  (Putnam,  1994). 

II.  Defining  subject-object  conceptual  distinctions 

Why  is  it  important  to  educational  research  and  instructional  design  to  address  philosophical 
issues  such  as  subjectivity  and  objectivity?  Eisner  & Peskin  (1990)  provide  this  answer:  Thoughts  have 
consequences;  how  we  think  about  subjectivity  and  objectivity  affects  research  procedure  because  these 
issues  are  typically  embedded  in  the  broader  framework,  abeit  most  often  implicitly,  that  directs  the 
conduct  of  our  inquiry  (p.  1 5). 

Careful  attention  must  be  given  when  discussing  the  notion  of  objectivity  and  subject-object 
distinctions,  since  the  terms  can  have  strikingly  different  connotations  depending  on  how  they  are  being 
used  and  to  what  they  refer.  The  subject-object  distinction  has  ontological,  epistemological,  and 
methodological  defining  levels,  along  with  relations  between  these  defining  levels.  The  ontological  level 
deals  with  assumptions  of  what  is  known  or  knowable  about  reality  and  its  nature.  The  epistemological 
level  address  the  relationship  between  the  knower  and  the  known  where  assumptions  concerning  this 
relation  depend  largely  on  the  ontological  features  supported.  The  methodological  level  deals  with  how 
one  goes  about  finding  things  out,  the  process  of  which  depends  largely  on  ontological  and  epistemological 
features  supported  (Guba,  1990). 

The  internal  realist  position  utilizes  a specific  formulation  of  the  subject-object  distinction  which 
respect  to  the  features  at  each  level  of  the  subject- object  distinction  described.  First,  in  terms  of  ontology, 


there  is  no  truth  claim  to  any  absolute  knowledge  that  subjects  have  of  objects  in  the  world  as  the 
Postpositivist/Realist  position  holds  Instead,  objectivity  becomes  a regulative  ideal.  Popper's  (1968) 
states: 

The  status  of  truth  in  the  objective  sense  , as  correspondence  to  the  facts  and  its  role  as  a regulative 
principle,  may  be  compared  that  of  a mountain  peak,  which  is  permanently,  or  almost  permanently, 
wrapped  in  clouds.  The  climber  may  not  merely  have  difficulty  getting  there  , because  he  may  be  unable  to 
distinguish,  in  the  clouds,  between  the  main  summit  and  some  subsidiary  peak.  Yet  this  does  not  affect  the 
objective  existence  of  the  summit.  . .The  very  idea  of  error,  or  of  doubt.  . implies  the  idea  of  an  objective 
truth,  which  we  may  fail  to  reach  (p.  226). 

Under  Popper's  reading  of  science,  what  is  considered  to  be  objectively  true  extends  beyond  what 
is  empirically  found  and  with  it  scientific  inquiry  entails  more  than  compiling  an  inventory  of  cold  hard 
empirical  facts.  This  interpretation  of  the  fundamental  aims  of  science  relocates  the  focus  of  objective  truth 
from  objective  facts  to  a larger  landscape  of  scientific  inquiry  more  congruent  with  the  extensive  range  of 
possible  human  experience  under  investigation. 

This  has  an  important  consequence  for  education's  use  of  scientific  methodology.  It  suggests 
what  we  already  know  to  be  true,  there  is  a lot  more  going  on  in  the  realm  of  experience  at  any  given  time 
than  scientific  inquiry  is  able  to  discern  and  that  trying  to  maintain  an  awareness  of  the  expansive  landscape 
of  experience  leaves  the  door  open  to  discovering  new  things  in  the  study  of  learning  processes  that  are 
continually  immerging  in  lives.  Closing  the  door  on  what  is  not  immediately  distinguishable  is  the  mark  of 
an  impatient  scientist 

Second,  in  terms  of  epistemological  features,  the  denial  of  the  conceptual  independence  of  subject  and 
object  is  not  a denial  of  the  possibility  of  human  subjects  achieving  true  knowledge  about  what  is 
objectively  there  in  actuality.  Putnam  (1987)  states,  "Kant's  glory  in  my  eyes,  is  to  say  that  the  very  fact 
that  we  cannot  separate  our  own  conceptual  contribution  from  what  is  'objectively  there'  is  not  a disaster.” 
The  fact  that  individuals'  experiences  take  place  in  the  world  and  within  social/cultural  communities  does 
not  in  any  way  detract  from  the  actual  distinction  that  exists  between  the  subject  experiencing  and  the 
object  experienced. 

Third,  in  terms  of  methodological  features,  the  individuals’  within  society  constitute  a community  of  critical 
inquirers.  Popper  expresses  this  well  when  describing  the  critical  spirit  of  objectivity  applied  to  science: 
The  objectivity  of  science  is  not  a matter  of  the  individual  scientists  but  rather  the  social  result  of  their 
mutual  criticism,  of  the  friendly-hostile  division  of  labour  among  scientists,  of  their  co-operation  and  also 
of  their  competition.  For  this  reason,  it  depends  in  part,  upon  a number  of  social  and  political 
circumstances  which  criticism  possible,  (p.  95). 

The  subject-object  conceptual  position  of  the  internal  realist  is  to  be  distinguished  from  one 
commonly  discussed  within  the  area  of  Constructivist  instruction.  In  the  case  of  Constructivist  instructional 
theory,  most  Constructivists  accept  only  one  subject-object  distinction,  that  which  is  part  of  an 
epistemological  belief  system  within  the  subject's  experience  . Constructivists  generally  oppose  making 
the  ontological  subject-object  distinction,  interpreting  this  view  as  being  part  of  an  Objectivist  stance,  to 
be  distinguished  from  an  Constructivist  view  which  does  not  commit  to  such  a claim  (Guba,  1990; 
Jonassen,  1991).  To  illustrate: 


Table  1:  Positions  on  subject-object  conceptual  distinctions 


Domain 

Postpositivism 

Constructivism 

Alternative 

Ontology 

Realist  * 

Relativist 

Internal  realist  * 

Epistemology 

Objective  * 

Consensual/subjective  * 

Critical  perspective  * 

Methodology 

Verificatory 

Hermeneutical 

reconstructive. 

Rational 

Communication  * 

* upholds  subject- object  distinction 
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Upholding  a commitment  to  ontological,  epistemological,  and  methodological  varieties  of  the 
object- subject  distinction  establishes  a standard  that  Constructivist  instructional  theories  have  tended  to 
conflate  (Bopry,  1999;  Cunningham,  1991;  Cobb  & Yackel,  1996;  Jonassen,  1991). 

III.  What  Constructivism  in  education  is  not 

Constructivism  is  not  a type  of  learning,  nor  is  it  a teaching  methodology.  It  is  not  to  be  taken  as 
some  learning  strategy  that  can  be  applied  one  day  in  a specific  context  and  then  forgotten.  Rather, 
Constructivism  can  be  categorized  as  a philosophy  of  learning  that  refers  to  how  individuals  learn  all  the 
time.  That  is,  individuals  constantly  construct  their  learning,  whether  they  are  actively  pursuing  some 
form  of  discovery  learning  or  sitting  in  a classroom  taking  notes. 

However,  there  are  different  types  of  learning  construction  (Clemens,  1 997;  Devries  & Zan,  1 996; 
Johnson,  1998)  for  different  levels  of  meaning  construction  (Bunge;  1979,  Gabora,  1997,  Luhman,  1986). 
Some  learning  processes  will  be  more  individually  centered  and  some  more  group  oriented.  In  addition, 
not  all  learning  processes  will  occur  at  the  same  level  of  individual  awareness  or  self-consciousness 
(Damasio,  1999;  Dennett,  1991).  There  is  no  one  way  that  has  succeeded  in  explaining  the  many  types  of 
learning  constructions  adequate  to  provide  a comprehensive  picture  the  Constructivist  learning  taking 
place. 

IV.  Statement  of  problem 

It  is  argued  that  the  conflation  of  the  ontological  subject- object  conceptual  distinction  has 
narrowed  the  range  of  what  the  epistemological  and  methodological  subject -object  conceptual  distinctions 
refers.  It  is  this  narrowing  of  the  range  of  inquiry  that  is  believed  responsible  for  conflicts  in  how 
instructional  designers  conceptualize  group  and  individual  oriented  instruction  , giving  rise  to  an  ongoing 
struggle  referred  to  here  as  the  paradox  of  Constructivist  instruction. 

V.  Objectives 

The  paper  draws  on  contemporary  work  in  neuroscience,  cybernetics,  communications  theory,  and 
philosophy  of  science  in  order  to  develop  a Communicative  Constructivist  Perspective  (CCP)  within  a 
framework  that  suggests  the  complementarity  and  parellel  duality  (independent  yet  integrated)  of 
Constructivist  learning  processes  and  knowledge. 


VI.  Constructivist  learning  processes 

One  way  to  view  the  Constructivist  learning  is  to  describe  it  in  terms  of  processes  that  make  it 
up  and  that  vary  in  level  of  conscious  awareness.  This  is  an  important  component  of  Constructivist 
instruction  in  that  research  on  learning  processes  informs  on  instructional  design.  What  is  striking  in  the 
Constructivist  literature  is  how  different  the  range  of  explanations  for  learning  processes  can  be  as  a 
function  of  the  Constructivist  position  held. 

i.  Meta-cognitive,  cognitive,  motivational  and  affective  processes 

Breadth  and  depth  of  self-system  processes  are  demonstrated  in  basic  psychological  processes 
identified  in  cognitive  psychology.  Meta-cognition  refers  to  the  monitoring  and  controlling  of  cognition 
(Zimmerman,  Bandura,  Martinez-Pons,  1992).  This  is  typically  differentiated  from  regular  cognitive 
processes.  Cognitive  processes  refer  to  those  mental  operations  required  to  encode  and  retrieve 
information.  Wolters  (1998)  views  cognitive  processes  as  goal  oriented,  strategic,  and  attentional. 
Motivational  processes  refer  to  the  underlyi  ng  drive  or  desire  towards  something.  Motivation  al  processes 
are  associated  with  the  directing  of  effort  towards  some  goal,  either  intrinsically  or  extrinsically 
determined.  Affective  processes  are  distinguished  from  cognitive,  meta-cognitive,  and  motivational 
processes.  Affect  has  been  used  to  refer  to  feeling  states,  moods,  and  emotional  experiences  (Newman, 
1994). 
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In  contrast  to  motivational  states,  affective  processes  often  tend  to  be  more  diffuse  and  sometimes  without 
any  specific  direction  or  focus.  The  diffuseness  of  certain  psychological  processes  (i.e.,  feeling,  emotion) 
have  made  it  difficult  for  cognitive  psychology  to  explain.  This  is  due  to  fne  fact  that  affective  processes 
are  largely  co-determined  processes  within  the  self-system  independent  of  self-consciousness,  making 
these  processes  difficult  for  individuals  to  grasp. 

ii.  Individual  Constructivism  and  volitional  processes 

Different  views  on  how  to  treat  intentional  and  collaborative  learning  have  Views  of  the  self  (I)  as 
a substantive  category  began  with  the  Cartesian  theory  (Descartes,  1641).  While  the  view  was  greatly 
criticized  throughout  modem  philosophy,  substantive  notions  of  the  self  continue  to  flourish.  McCombs  & 
Marzano  (1990)  view  the  personal  self- structure  as  the  generative  structure  for  self-processes.  They  state, 
"To  generate  the  will  for  self-regulation,  students  must  realize  that  they  are  creative  agents,  responsible  for 
and  capable  of  achieving  self-development  and  self-determination  goals,  and  they  must  appreciate  and 
understand  their  capabilities  for  reaching  these  goals."  This  gets  at  the  importance  of  individual  ’self- 
directedness'  quality  of  learning. 

Von  Glaserfeld  (1995)  labels  volitional  processes  as  an  individual's  'mapping  of  actions  and 
conceptual  operations  that  had  proven  viable  in  the  subject’s  experience.  Deci,  Ryan,  & Koestner's  (1 999) 
theory  of  self-determination  is  concerned  with  the  degree  to  which  individuals  experience  their  mental 
processes  to  be  freely  chosen  rather  than  being  coerced  by  desired  outcomes.  They  distinguish  self- 
determination  from  external  determination  by  the  extent  to  which  individuals  believe  they  have  causal 
control.  At  this  level,  self-determining  processes  are  connected  to  the  concept  of  "self."  Such  processes 
take  place  at  the  level  of  self-consciousness,  but  there  are  other  processes  that  cannot  be  explained  within 
the  framework  of  the  "self'  concept.  This  is  the  case  for  different  levels  of  processes  where  self- 
consciousness  does  not  accompany  the  processes  taking  place. 

iii.  Social  Constructivism  and  social  processes 

Social  processes  have  also  been  implicated  in  the  determination  of  constructivist  learning.  Much 
of  the  work  in  this  area  is  referred  to  as  social  Constructivism,  drawing  its  main  theoretical  basis  from 
Vygotsky  (1987),  social  activity  theorists  (Bordieu,  1976;  Garrison,  1998;  Lave  & Wenger,  1991),  and 
pragmatism  (Rorty,  1978;  Putnam,  1987).  Garrison’s  (1996),  approach  emphasizes  individual's  self- 
realization  being  derived  from  actions  in  the  social  world.  Social  processes  are  largely  embedded  in  a social 
context  characterized  by  argument,  discussion  and  debate. 

Hare  (1983)  supports  the  position  that  individual  processes  are  social  in  origin  and  create  the 
various  unities  of  personal  identity  (sense  of  personal  identity,  self-consciousness,  agency).  The  ability  to 
conceive  of  oneself  as  a unique  singular  being  is  a necessary  precondition  for  the  acquisition  of  a theory  of 
self,  experienced  as  one's  sense  of  identity.  Self-consciousness  involves  both  knowing  what  one  is 
experiencing  (consciousness)  and  that  one  is  experiencing  it,  which  involves  the  capacity  (concept  of 
theoretical  self)  to  be  able  to  make  some  form  of  self-reference.  Under  this  view  the  primary  human  reality 
of  social  processes  is  taken  to  be  persons  in  conversation.  Hare  (1983)  states,  "The  psychological 
secondary  structure  is  a reflection  of  the  primary  structure,  the  array  of  persons  and  their  conversation 
which  is  the  primary  reality  of  the  society  which  brings  them  into  being." 

VII.  Constructivist  learning  knowledge 

Constructivist  learning  can  also  be  described  in  terms  of  the  knowledge  structures  and  capacities 
attributed  to  it.  Goldstein  (1986)  refers  to  knowledge  as  ’an  organized  body  of  knowledge  usually  of  a 
factual  or  procedural  nature,  which,  if  applied  makes  adequate  job  performance  possible.’ 

In  this  discussion  on  Constructivist  Instruction,  knowledge  is  used  to  refer  to  the  mental  facts, 
procedures,  and  strategies  individuals  rely  on  when  making  judgements  and  carrying  out  actions.  This  is  to 
be  distinguished  from  learning  processes  that  refers  to  how  individuals  acquire  knowledge  and  skills 
(Gordan,  1 994).  This  distinction  is  important  for  two  reasons.  First,  it  maintains  the  dichotomizing 
strategy  used  throughout  the  paper.  Knowledge  is  an  object  acquired  by  subjects  through  learning 
processes.  Second,  distinguishing  knowledge  from  learning  processes  can  allow  for  advances  in 
understanding.  Gordan  (1994)  states,  ’By  understanding  something  about  the  basic  types  of  knowledge  that 
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experts  and  novices  use  in  performing  tasks,  we  can  enhance  the  processes  involved  in  front  end  analysis, 
instructional  system  design  , and  program  evaluation.' 

Research  studying  the  nature  of  knowledge  and  how  it  is  acquired  has  given  rise  to  various 
taxonomies  being  used  to  provide  grounding  for  instructional  program  designs  (Anderson,  1 983;  Gagne, 
1985;  Rasmussen,  1986).  The  taxonomy  employed  in  this  exploratory  piece  deals  with  the  two  types  of 
knowledge  traditionally  associated  with  the  notions  of  objectivity  and  use  of  subject-object  conceptual 
distinctions,  These  are  scientific  knowledge  and  universal  knowledge. 

i.  Universal  knowledge 

Rationalist  philosophy  has  provided  initial  support  for  viewing  human  beings  as  possessing 
universal  knowledge  . Spinoza  (1677)  stated,"  Human  reason  begins  in  the  same  reason  with  its  native 
powers  and  thus  creates  its  first  intellectual  tools."  Similarly,  Piaget's  (1954)  cognitive  development 
approach  posited  universal  structures  of  knowledge  (e.g.  pre  logical,  concrete,  abstract)  or  general 
categories  that  evolved  with  the  biological  organism  (genetic  epistemology). 

Discourse  on  universal  knowledge  is  not  limited  to  individual  rational  principles  but  also  includes 
communicative  principles  which  govern  all  communicative  exchange.  Habermas'(1989)communicative 
approach  to  universal  knowledge  maintains  its  individual  appeal  to  rationality  while  being  at  the  same  time 
deeply  related  to  communicative  exchange  and  has  the  potential  to  offer  much  to  educational  researchers 
interested  in  the  theoretical  bases  of  the  relation  between  individual  and  social  learning. 

Habermas'  Discourse  Ethics  ( 1 989)  seeks  to  imbed  communicative  knowledge  within  a dialectical 
framework,  which  acts  as  the  moral  determinate.  He  accomplishes  this  by  treating  human  consciousness 
as  that  which  is  structured  by  language  exchange  within  a normative  structure  of  social  interactions. 
Habermas'  modified  version  of  universal  morality  can  be  characterized  by  the  following  features: 

1)  Habermas  advocates  a communicative  theory  of  meaning  where  validity  and  truth  claims  are 
decided  by  resolving  normative  rightness,  which  can  be  determined  through  discursive  argumentation. 

2)  Habermas  (1990),  summarizes  the  generalized  imperative  that  corresponds  to  his  theory  of 
argumentative  discourse.  He  states,  "All  affected  can  accept  the  consequences  and  the  side  effects  its 
general  observance  can  be  anticipated  to  have  for  the  satisfaction  of  everyone’s  interests  (and  these 
consequences  are  preferred  to  those  of  known  alternatives  possibilities  for  regulation)." 

3)  The  justification  of  Habermas'  universal  morality  lies  in  accepting  universality  as  a procedural 
principal  of  practical  discourse.  Habermas'  notion  of  universality  ('U')  requires  that  each  individual  adopts 
the  perspective  of  all  others  that  are  affected  by  the  consequences  of  argumentative  discourse.  The  types  of 
questions  that  can  be  treated  in  such  a manner  are  those  that  concern  rightness  and  just  regulation  of  social 
interactions  involving  all  persons. 

For  Habermas,  moral  practices  are  social  matters  to  be  decided  by  discourse  interactions  of 
individually  deliberating  subjects.  Thus,  both  individual  will  and  community  practices  are  taken  into 
consideration  by  Habermas'  universal  theory  of  argumentative  discourse.  Habermas  supports  the  causal 
role  of  socialization  in  shaping  personal  identity  as  well  as  the  capacity  of  discourse  to  represent  this. 

ii.  Scientific  knowledge 

The  assumption  at  the  beginning  of  this  paper  attributed  to  scientific  inquiry  the  notion  of 
objectivity  construed  as  a regulative  principle  under  the  Internal  Realist  stance.  Now  the  question 
becomes, what  is  the  nature  of  scientific  knowledge  within  the  philosophical  framework  of  Internal 
Realism?  The  following  answer  is  provided. 

At  the  level  of  basic  neurophysiological  organization  there  is  multiplicity  in  function.  The  human 
brain  and  its  respective  components  that  underlie  all  cognitive  function  are  not  simply  limited  to  specific 
functions.  Edelman  (1989)  states,  "There  is  no  unique  structure  of  combination  of  groups  responsible  to  a 
given  category  or  pattern  of  output.  Instead,  more  than  one  combination  of  neuronal  groups  can  yield  a 
particular  output , and  a given  single  group  can  participate  in  more  than  one  kind  of  signaling  function." 

Even  at  the  most  rudimentary  level  of  the  human  biological  organisms,  it  is  recognized  that  there  is  an 
interactive  learning  process  taking  place  where  some  neuronal  pathways  are  stimulated  and  strengthened 
with  ongoing  stimulation,  whereas  others  are  not.  Dennett  (1991)  refers  to  this  as  the  plasticity  in  nervous 
systems.  This  multi-function  capacity  could  be  used  to  help  explain  intercultural  and  interpersonal 
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differences  reflected  in  individuals'  learning  styles  with  respect  to  the  various  backgrounds  from  which  they 
evolve. 

In  addition  to  multi-functioning  capacities, scientific  knowledgehas  been  attributed  a multi- 
perspective quality.  Science-based  knowledge  is  useful  in  providing  leverage  for  multiple  perspective 
explanations.  Bohr’s  ’’complementarity  theory”  involves  particles  and  waves  while  Aschby's  "Law  of 
Requisite  Variety"  provide  strong  support  for  the  complementarity  of  multiple  perspectives  (Boyd  & 
Zeman,  1 995).  The  popularity  of  the  Kuhnian  notion  of  "paradigms"  in  historical  scientific  inquiry  and 
recent  scientific  work  attest  to  the  recognition  of  multiple  perspectives  (Horwich,  1 993).  Assumptions  of 
complementarity  of  perspectives  pull  together  evolutionary,  genetic,  and  rational  perspectives  on  universal 
knowledge.  This  is  an  alternative  to  extreme  positions  which  support  either:  1 ) there  being  only  one 
accurate  description  of  reality  possible  (Scientific  Realism)  or,  2)  there  being  no  accurate  description  of 
reality  possible  (Postmodernism).  This  could  prove  useful  in  promoting  interdisciplinary  approaches  to 
learning  with  a critical  orientation. 

Scientific  knowledge  informs  on  Constructivist  instructional  theory  in  multiple  ways.  First, 
knowledge  gained  from  neurological  research  informs  on  how  the  brain  structures  work  , how  they 
develop,  and  how  they  can  be  changed  (Edelman,  1 992).  This  contributes  by  providing  information 
concerning  how  the  mind  functions.  Edelman  (1992)  states, ’’A  description  of  mind  cannot  proceed 
liberally-that  is,  in  the  absence  of  a detailed  biological  description  of  the  brain."  It  is  a limitation  that 
cognitive  science  has  traditionally  adopted  a functionalist  position  defining  the  mind  as  being  made  up  of 
mental  representations  that  operate  according  to  a set  of  definite  procedures  or  computational  functions  that 
can  be  studied  independently  of  underlying  structure. 
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VIII.  The  struggle  for  a framework  to  accommodate  Constructivist  learning  processes  and 
knowledge 


In  order  for  a more  complete  grasp  of  learning  to  be  achieved,  there  has  to  be  some  way 
to  make  intelligible  all  relevant  knowledge  structures  and  mental  processes  that  constitute  it.  Markus  & 
Wurf,  (1987)  insist  on  the  need  for  the  integration  of  the 

complex  set  of  intrapersonal  (affect,  cognition,  motivation)  and  interpersonal  (social  perceptions, 
feedback  from  others)  processes. 

Efforts  to  describe  learning  processes  involved  in  accessing  knowledge  have 
resulted  in  the  positing  of  various  descriptions  of  mental  processes  and  mental  models.  Johnson- 
Laird  (1983)  states,  'Mental  models  enable  individuals  to  make  inferences  and  predictions,  to  understand 
phenomena,  to  decide  what  action  to  take  and  to  control  its  execution,  and  above  all,  to  experience  events 
by  proxy  (p.  397).'  Understanding  the  relationship  between  knowledge  taxonomies  and  corresponding 
mental  models  is  a central  part  of  instructional  program  design  and  its  description.  Therefore,  the  use  of 
information  technologies  and  communication  theory  in  the  following  discussion  addresses  both  functional 
and  structural  aspects. 

Insights  from  neurological  research  recommend  greater  attention  be  paid  to  human  functions  that 
have  been  difficult  to  address  by  concentrating  on  psychological 

processes  alone  (i.e.,  affect).  Demonstrating  the  breadth  and  depth  of  processes  requires  a set  of 
knowledge  tools  to  get  at  those  processes  that  fall  beyond  methodological  approaches  employed  in 
cognitive  psychology  . Damasio  ( 1 998)  supports  there  being  both  a biologically  'core  consciousness'  that  is 
relatively  stable  across  one's  lifetime  as  well  as  an  ’extended  consciousness’  with  many  levels  and  grades 
that  produce  an  elaborate  sense  of  self  through  lifelong  individual  processes.  In  this  way,  what  counts  as 
constructivist  learning  can  be  seen  to  extend  beyond  what  individuals  are  consciously  aware. 

i.  Contributions  from  Cybernetics 

A recent  turn  in  Constructivist  instruction  to  the  Cybernetic  science  offers  new  insights  for 
instructional  designers.  Basic  Cybernetic  systems  are  organizing  systems,  operating  by  feedback 
mechanisms  mediating  from  system  outputs  to  subsequent  system  inputs.  Bopry  (1999)  supports  that  this 
turn  has  the  potential  to  provide  Constructivist  practitioners  a 'philosophical  mooring  within  the  field  itself.’ 
Developments  in  this  field  have  yielded  numerous  models  of  autonomous  systems  functions  used  to 
describe  the  operation  of  living  and  non-living  systems  (Varela,  1981).  Basic  autonomous  systems  can  be 
characterized  by  four  fundamental  features:  1)  organizational  closure,  2)  structural  determination,  3) 
structural  coupling,  4)  proscriptive  development. 

Organizational  closure  refers  to  the  organizing  of  the  defining  relations  of  a system  necessary  for 
the  system  to  exist.  Varela  (1981)  states  "Organizational  closure  arises  through  the  circular  concatenation 
of  processes  to  constitute  an  interdependent  network.”  Without  organizational  closure,  autonomous 
systems  could  not  exist.  In  the  game  of  chess,  were  it  not  that  each  chess  piece  had  its  respective 
operations  on  the  chessboard,  chess  could  not  exist.  Structural  determination  refers  to  the  internal 
dynamics  of  autonomous  systems  responsible  for  structural  change.  Autonomous  systems  are  limited  by 
the  interactions  that  its  structure  makes  possible.  Humans  do  not  have  the  necessary  wing  structure  to  fly 
as  do  birds.  However,  human  structural  determination  does  allow  for  walking.  Structural  coupling  refers 
to  the  interactions  that  autonomous  systems  have  with  the  environment.  Bopry  (1999)  states,  "When  a 
unity  is  in  continuous  interaction  with  the  environment,  so  there  is  a mutual  triggering  of  structural  change 
over  time  that  is  stable  in  nature,  the  unity  and  the  environment  are  said  to  be  structurally  coupled." 
Maturana  & Varela  (1987)  support  that  structural  coupling  represents  the  basis  for  higher  order  cognitive 
development.  Proscriptive  development  describes  how  it  is  that  nature  constrains  living  organisms  during 
the  process  of  evolutionary  change.  Varela  (1991)  states,  "In  a proscriptive  context  natural  selection  can  be 
said  to  operate,  but  in  a modified  sense:  selection  discards  what  is  not  compatible  with  survival  and 
reproduction.” 

Proscriptive  functions  are  not  limited  to  explanations  of  biological  evolution.  Bopry  (1999) 
connects  proscriptive  development  with  culture  and  language  use.  Bunge's  (1977)  emergent  level 
cybernetic  theory  and  Boyd's  (1993)  cybersystemic  theory  provide  explanations  for  multiple  levels  of 
emergent  processes  that  include  neurophysiological,  autobiographical,  and  psycho-social  processes. 
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Damasio's  (1998)  linking  of  conscious  processes  to  neural  architecture  supports  a general  anatomy  of 
consciousness. 

Cybernetic's  attention  to  the  role  of  structure  and  organization  in  structural  change  runs  concurrent 
with  the  underlying  assumptions  of  this  paper  pertaining  to  subject-object  conceptual  distinctions.  The 
subject-object  conceptual  distinction  reappropriated  as  a Cybernetic  structural -organizational  conceptual 
distinction  is  supported  by  views  of  the  commensurability  of  universal  and  scientific  knowledge. 

From  an  evolutionary  perspective,  a universal  drive  for  self-preservation  is  recognized  that  binds 
individuals  together  in  communicative  collectives  (Gould,  1986;  Dawkins,  1973).  Evolutionary  theory 
supports  the  universal  human  drive  to  create  (propagate)  one’s  self  (Darwin,  1871),  one's  genes  (Dawkins, 

1 976),  or  one's  mental  representations  or  'memes'  (Blackmore,  1 999).  Cybernetics  and  evolution  theory 
provide  support  for  viewing  human  beings  as  possessing  universal  knowledge  tools  (Blackmore,  1999; 
Boyd  & Zeman,  1993). 

To  take  this  one  step  further,  Boyd  (1993)  supports  the  aesthetic  critique  as  a universal  knowledge 
tool.  Advancements  in  aesthetic  education  have  their  biological  origins  in  early 
attractiveness/repulsiveness  experiences  with  the  world.  This  basic  level  of  biologically  evolved 
perceptual  engagement  forms  the  basis  of  what  later  may  develop  into  personalized  and  socialized 
preferences.  Gabora  (1997)  supports  that,  in  line  with  evolutionary  theory,  mental  representations  (or 
memes)  evolve  through  adaptive  exploration  and  transmission  of  information  by  way  of  variation  and 
selection.  This  can  be  employed  to  explain  the  evolution  of  culture  and  creativity.  Boyd  & Zeman’s 
(1993)  notion  of  "generative  concepts"  is  treated  as  a set  of  actively  developing  tools  that  function  on  a 
meta-level  as  principles  for  conceptual  organization. 

Cybernetics  has  the  potential  to  make  two  important  contributions  to  Constructivist  Instrional 
theory.  First,  it  gets  at  a depth  of  Constructivist  learning  that  has  been  neglected  in  contemporary 
instructional  design  theory.  This  can  be  used  to  provide  a basis  for  inquiry  into  a greater  range  of  learning 
processes  than  has  been  considered  in  instructional  theory.  Second,  there  is  also  an  important  recognition  of 
the  genetic  epistemological  structures  that  have  a causal  role  within  the  complex  set  of  learning  processes 
that  instructional  designers  are  interested  in.  The  potential  for  the  inclusion  of  science  is  quite  attractive  for 
instructional  theories  like  Constructivist  that  do  not  provide  obvious  "self-correcting"  mechanisms  for 
interventions  designed. 

ii.  Recursive  communication 

Krippendorff  (1994)  advances  a recursive  theory  of  communication  based  on  assumptions  of  the 
self-referential  quality  of  human  communication.  This  approach  to  human  communication  focuses  on  the 
process  of  communicating  as  well  as  what  is  communicated.  It  puts  forth  the  following  propositions: 

1)  Human  communication  theory  must  also  be  about  itself. 

2)  Everything  said  is  communicated  to  someone  understanding  it  as  such. 

3)  Human  communication  constitutes  itself  in  the  recursive  unfolding  of  communication  constructions,  held 
by  participants  (including  of  each  other),  into  intertwining  practices  that  these  participants  can  recognize 
and  explain  in  terms  of  being  in  communication. 

This  approach  to  human  communication  contains  two  defining  features  that  are  crucial  to 
Constructivist  Instructional  design.  First,  it  acknowledges  self-referential  quality  of  experience.  Asserting 
that  communication  theory  is  about  itself  is  to  recognize  that  individuals  experiences  (even  acts  of 
theorizing  about  communication)  are  not  products  of  the  outside  world  but  rather,  are  constructed  from 
within  the  realm  of  one's  own  experiences.  Krippendorff  (1995)  states, 'Whatever  gives  rise  to  the 
awareness  of  something  being  said  and  communicated,  the  causes  of  ones  experiences,  must  be  located 
within  one's  horizon  od  understanding.’  As  such,  individuals  are  responsible  for  constructing  their  own 
communication  and  the  communication  of  others 

Second,  it  recognizes  the  recursiveness  of  human  experience.  Individuals  monitor  their 
communications,  transforming  the  consequences  of  actions  into  information  that  revises  knowledge  used  to 
direct  future  actions.  It  maintains  the  necessary  positioning  of  oneself  within  communications  which 
includes  other  human  beings  and  to  attempt  to  understand  others  perspectives. 

Together,  Cybernetics  and  Recursive  Communication  Theory  represent  innovative  approaches  for 
the  linking  of  Constructivist  learning  processes  and  knowledge  under  a complementary  dualistic 
framework.  Instead  of  conflating  object-subject  distinctions,  this  view  suggests  that  it  is  possible  to  make 
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connections  between  complementary  knowledge/structures  and  processes/functions  that  advance 
understanding.  To  illustrate: 

Table  2:  A complementary  framework  for  Constructivist  learning  processes  and  knowledge 
Constructivist  Learning  Processes 


Constructivist  Learning 

Processes  -Knowledge 

Constructivist  Learning 
Knowledge 

Individual  Constructivism 

Cybernetics 

Scientific  Knowledge 

Social  Constructivism 

Recursive  Communication 

Universal  Communicative 
Knowledge 

The  Communicative  Constructivist  Perspective  ( CCP)  presented  next  takes  the  resolution  of  the  Paradox  of 
Constructivist  instruction  to  be  its  primary  focus.  It  does  so  largely  by  drawing  together  the  theoretical 
strands  discussed  thus  far. 

IX.  Communicative  Constructivist  Perspective  (CCP):  A multi-level,  multi-perspective  account 

CCP  is  proposed  to  describe  the  breadth  and  depth  of  Constructivist  learning  experienced  by 
individuals  living  within  a community  of  learners.  The  first  fundamental  criterion  concerns  the  importance 
of  viewing  meaning  construction  from  multiple  perspectives  (aspects)  and  sharing  these  perspectives  in 
such  a way  as  to  shape  individual  and  collaborative  learning.  This  gets  at  the  need  for  both  individual 
expression  and  collaborative  communication  where  individual  expressions  occur  and  develop  within 
individuals'  sharing  of  subjective  experiences. 

In  practical  terms,  students'  learning  experiences  do  not  take  place  in  isolation.  Learning  involves  all 
individuals  who  partake  in  the  ongoing  communication  and  decision  making  together.  This  minimally 
includes,  learners,  parents,  teachers,  and  administration  coming  together  to  express  views  in  ongoing 
discussions. 

The  second  fundamental  criterion  of  CCP  concerns  employing  definition  of  Constructivism  that  is  broad 
enough  to  capture  the  multiplicity  of  knowledge  structures  (Boyd  & Zeman,  1993;  Cobb  & Yackel,  1996; 
Edelman,  1 989;  Hare,  1 983)  at  the  various  levels  of  meaning  production  (biological , psychological,  social) 
that  influence  learning  experiences.  CCP  addresses  structural  knowledge  and  self-systems  processes  that 
have  been  identified  both  outside  and  within  Constructivist,  Situated-Cognition,  and  Self-regulated 
learning  literatures  (Cobb,  1994;  Yang,  1993;  Zimmerman,  Bandura,  & Martinez-Pons,  1992). 

In  practical  terms,  learning  is  assumed  to  be  multi-perspectual  and  therefore,  requires  flexible  multi-level 
knowledge  to  be  accommodated  within  higher-order  learning  activities  in  order  to  explain  its  complex 
nature  within  a community  of  learners.  This  is  essential  in  interdisciplinary  programs  of  educational 
instruction  where  students  are  exposed  to  a diverse  range  of  educational  content. 

When  designing  education,  it  is  important  to  recognize  that  what  constitutes  individual  learners 
extends  beyond  psycho-social  processes  and  our  sense  of  self.  In  order  to  develop  individual  and 
collaborative  learning,  efforts  should  be  directed  at  both  the  learner  and  the  learning  environment.  For  this 
reason,  educational  design  requires  theoretically  grounded  proscriptive  and  prescriptive  necessary 
conditions.  Utilizing  a CCP  for  the  purpose  of  educational  design  could  result  in  adopting  the  following 
orientation: 


Table  3:  Key  Postulates  of  the  CCP  Perspective 


CCP  Postulates 

Description 

Critical  Orientation, 

Not  all  subjectively  constructed  meaning  will  be  equally  accurate  and  it  is  an  asset 
to  be  able  to  critically  evaluate  learning  constructions. 

Process  and  Identity 
Orientation. 

Because  this  real  world  is  subjectively  experienced  by  each  individual  within  a 
social  realm,  there  is  a dual  need  to  develop  one's  own  learning  processes  and 
personal/social  identity. 

Multiple- 

Perspectives 

Many  viewpoints  or  perspectives  contribute  to  a more  complete  understanding 
which  more  closely  approaches  the  truth. 
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Communicative 

Constructivist  learning  often  involves  students,  parents,  teachers,  and  all  other 

Orientations 

stakeholders. 

This  draws  together  much  of  the  Constructivist  learning  mentioned  already  in  an  effort  to  apply  it 
to  individually  meaningful  collaborative  learning  process.  Some  of  postulates  (i.e.,  authentic  and 
collaborative  learning,  decision  making,  communication  skills  development,  etc.). 

The  CCP  general  orientation  outlined  could  contribute  to  education  applied  to  the  design  of 
interventions  in  order  to  modify  the  structuring  and  content  of  instruction  in  an  effort  to  raise  individuals' 
awareness  of  humans  complex  organisms  amongst  other  complex  organisms  interconnected  at  various 
levels  within  the  learning  environment. 

X.  Educational  and  scientific  contribution 

Overall  the  notion  of  objectivity  assumed  to  be  a regulative  principle  was  supported  by  efforts 
throughout  the  paper  to  demonstrate  the  complementarity  of  Constructivist  Instruction  informed  by  science 
and  communicative  theory. 

First,  it  could  offer  a causal  account  of  learning.  Causal  explanations  are  well  suited  for  explaining  general 
measures  of  student  functioning  such  as  attitude  and  achievement  (Zimmerman,  1 986).  There  are  very  few 
causal  explanations  to  be  found  in  Constructivist  or  self-regulated  learning  literatures  (McCombs  & 
McCombs,  1990).  Causal  accounts  have  an  explanatory  value  that  could  provide  a great  contribution  to 
the  Constructivist  literature  if  fully  developed. 

Second,  it  could  act  as  a multi-level  explanation,  addressing  learning  at  multiple  levels  where  it 
occurs  (physiological,  psychological,  social).  This  is  an  emergent-level  perspective  in  that  there  are 
simultaneous  levels  of  learning  emerging  at  the  same  time  with  their  own  respective  properties  (Bunge, 
1979).  However,  it  is  also  a causal  explanation,  maintaining  the  subject-object  separation  that  other 
accounts  (i.e.,  symbolic  interactionism)  do  not  uphold. 

Third,  CCP  could  be  used  as  a multi-function  explanation,  concerned  with  the  necessary 
flexibility  of  learning  (i.e.,  learner  style,  self-efficacy,  intention,  etc.).  It  could  also  include  consideration  of 
contextual  functioning  (i.e.,  meta-cognitive  and  cognitive  strategic  planning,  goal-orientation,  learner 
control,  etc.). 

Fourth,  it  could  be  used  to  uphold  the  distinction  between  knowledge  structures  and  processes 
that  underlie  individuals'  psychological  processes  and  abilities  to  be  self-determined  learners  within  a 
community  of  learners.  Being  able  to  attend  to  both  knowledge  and  processes  is  considered  essential  to 
achieving  a more  complete  grasp  of  Constructivist  learning.  The  strategic  integration  of  these  elements 
offers  a philosophical  'mooring'  for  Constructivist  Instructional  Theory  and  Practice  . 

An  objective  communicative  set  of  procedures  can  also  address  standards  of  evaluation  required 
for  effective  instruction.  The  contribution  to  be  made  lies  in  how  it  is  that  standardized  evaluative 
measures  essential  to  instruction  are  treated.  Under  this  view,  problems  of  evaluation  are  resolved  by 
Constructivist  instructions'  prescriptive  function.  First,  evaluations  would  not  simply  be  administered  but 
would  be  integrated  as  part  of  the  learning  process.  This  can  be  accomplished  by  making  clear  who  is 
responsible  for  creating  the  evaluative  standards  and  when.  This  way  students  can  feel  they  are  not  merely 
subjected  to  some  imposed  standard,  but  rather  are  participating  in  the  standard  evaluation.  This  is  done  so 
that  students  can  learn  to  understand  the  standard  as  a first  step  in  being  able  to  participate  in  the  evaluation 
and  selection  of  future  standards.  This  can  be  taken  to  be  a type  of  cognitive  apprenticeship  (Clancy,  1992; 
Cobb,  1996;  Collins, 1991). 

Second,  learners  are  participating  in  standard  evaluations  administered  not  with  the  understanding  that  the 
standard  is  true  but  rather,  is  a logical  possibility,  objectively  true  for  all  learning  participants  and  to  be 
worked  towards  in  a cooperative  manner  (Habermas,  1995;  Kagan,  1990).  This  captures  the  essence  of 
what  Constructivism  should  encourage  when  attempting  to  provide  instruction  in  an  educational  setting. 

Conclusion 

The  multi-level  CCP  offered  an  alternative  to  the  paradox  of  Constructivist  instruction  by 
focusing  on  recent  developments  in  Cybernetics  and  communicative  theory  to  get  at  a depth  of 
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Constructivist  learning  neglected  in  contemporary  instructional  design  theory.  Support  was  given  to 
demonstrate  that  what  affects  each  of  us  as  learners  extends  beyond  our  psycho-social  processes.  The 
preseni  discussion  focused  on  these  underlying  elements  of ‘learning  in  demonstrating  their  potential 
implications  for  the  development  of  learning  interventions. 

Future  work  will  be  directed  at  developing  educational  interventions  that  raise  the  general 
awareness  of  the  complex  set  of  learning  processes  and  knowledge  that  arise  from  individual  and 
collaborate  Constructivist  instruction.  This  could  be  beneficial  in  promoting  self  aware  and  socially 
responsible  learning. 
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Using  SMIL  to  Create  a Web-Based  Multimedia 
Virtual  Library  Tour 

Yuwu  Song 

Arizona  State  University 


Abstract: 

This  article  shows  how  to  use  SMIL  to  create  an  Online  Multimedia  Virtual  Library  Tour.  The 
presentation  is  composed  of  a slide  show  of  text  and  still  images,  which  introduces  the  library.  It  is 
accompanied  by  background  audio  and  a caption.  In  this  presentation  there  are  hyper-links  to  related  library 
web  sites. 

URL:  htto://www, asu.edu/lib/webcom/vuwu/VisualTour/visualtouv.smil 

To  view ; this  file,  you  need  a RealPlayer  plug-in  which  can  be  downloaded  free  from: 
http://www . real,  com. 

The  Virtual  Library  Tour  presentation  is  created  by  using  Synchronized  Multimedia  Integration 
Language  (SMIL).  SMIL  as  a new  web  format  allows  us  to  put  text,  still  photos,  audio,  video,  and 
animation  into  interactive  presentations.  The  World  Wide  Web  Consortium  on  June  15,  1998 
( www. w3 . org/tr/rec- sm i 1)  published  the  specification  which  defines  SMIL  as  an  application  of  XML.  The 
specification  depicts  the  design  layout  and  temporal  behavior  of  multimedia  presentations. 

With  SMIL,  you  can  produce  interactive  multimedia  presentations  of  broadcast  quality.  You  can 
provide  links  within  a presentation  that  access  other  presentations,  other  media  files,  or  web  sites.  Before 
the  streaming  technology  came  into  being,  multimedia  files  such  as  MPEG  video  or  AIFF  audio  had  to  be 
downloaded  completely  before  we  could  see  them  play.  Streaming  technology  allows  multimedia  servers 
send  a file  in  a continuous  stream  that  can  be  played  back  shortly  after  the  local  machine  get  parts  of  the 
files.  It  would  take  a few  minutes  for  a traditional  video  file  to  be  downloaded  while  for  a streaming  video, 
it  only  takes  a few  seconds  before  you  can  view  it. 

To  create,  publish,  and  broadcast  SMIL  presentations,  all  you  need  is  a PC  and  access  to  a 
compatible  server.  To  see  SMIL  in  action,  you  need  a compatible  player  such  as  RealPlayer  G2  (free, 
downloadable  from  http ://www.real .com.)  Once  you  download  and  install  the  player,  you  can  view  SMIL 
presentations.  You  can  also  access  SMIL  presentations  from  the  Web.  When  you  do  this,  your  browser 
launches  the  SMIL  player,  which  in  turn  displays  the  presentation. 

Like  HTML,  SMIL  is  a subset  of  XML,  and  because  XML  is  defined  using  text  markup  tags,  you 
could  make  the  layout  and  design  of  SMIL  presentations  using  a regular  text  editor  such  as  NotePad. 

SMIL's  media  object  tags  allow  you  to  put  various  media  types  in  your  presentations,  and  they  are 
self-explanatory.  The  <audio>  tag  supports  audio  files  such  as  AIFF.  The  <video>  tag  supports  video 
formats  such  as  RM  files  . The  <img>  tag  is  for  still  images  such  as  JPEG  files  or  RealPix.  The  <text>  tag 
is  for  static  text  files;  the  <text  src>  or  <textstream>  tag  is  for  streaming  text  files  such  as  RealText.  You 
can  mix  any  of  the  available  media  types  in  your  presentations. 

The  presentation  layout  determines  how  you  set  up  your  screen.  SMIL  players  have  a window.  When 
you  want  to  play  multiple  media  types  simultaneously,  you  need  to  create  regions  or  mini-windows  within 
the  main  window. 

Regions  are  like  cells  within  a HTML  table.  You  can  decide  the  size  and  position  of  the  window 
as  well  as  the  size  and  position  of  the  mini -windows  within  it.  The  root-layout  sets  the  height  and  width  of 
the  entire  presentation  in  pixels.  This  is  the  whole  player  area,  which  will  be  divided  into  regions.  Each 
region  sets  specific  areas  in  the  presentation  that  media  will  play  in. 

<layout> 

<root-layout  height="405"  width-770"  background-color="black7> 

<region  id- ’title"  left="5"  top-’ 120"  width="400"  height="200”  z index- T7> 
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<region  id- "full"  lefp=  "0 " top="0"  height="425"  width="450" 
background-color="#602030"/> 

<region  id="video"  left=’'450"  top=’'240"  height*"  1 20"  width="315H  z-index=nii7> 
<region  id="toc"  left="450"  top=,'0”  height="405n  width- *2557> 

</l  ayout> 


First  we  size  the  main  window  (identified  with  the  <root-layout>tag)  to  405  by  770  pixels  and 
then  create  4 mini-windows  within  the  main  window:  title,  full,  video,  and  toe.  The  title  region  is 
positioned  5 pixels  to  the  right  of  the  main  viewing  area’s  left  edge  and  120  pixels  down  from  the  top  of  the 
main  viewing  area.  The  other  regions  are  placed  according  to  their  respective  pixels.  By  defining  layout 
regions,  you  can  combine  multiple  forms  of  media  and  display  them  simultaneously.  The  following  picture 
will  show  you  how  the  interface  looks  like. 


Let’s  take  a look  at  how  SMIL  combine  media  files  and  make  them  play.  The  following  code 
shows  an  example  of  event  timing: 

<par> 

<audio  src=http://www. asu.edu/lib/webcom/yuwii/VisualTour/smone2.rmi 
dur=”400s7> 

<text  src="http://www. asu.edu/lib/webcom/yuwu/VisualTour/toc.rt1’  region=toc”/> 

<text  src=”http://www,asu.edu/lib/webcom/yuwu/VisualTour/song.rt"  region- 'video’7> 

<seq> 

<!—  This  part  displays  the  title  screen  and  the  caption  with  an  audio  soundtrack-> 

<text  src=”http://www.asu.edu/lib/webcom/yuwu/VisualTour/title.rt” 
type- 'text/htm!”  region=”titie"  dur="35s7> 
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<!--  This  section  displays  a slide  show  — > 


<par> 


<img  src=http://www. asu.edu/lib/webcom/yuwu/VisualTour/map.rp 
region- 'full"  fill- ,freeze7> 


</par> 

<!—  This  section  contains  the  annotated  slideshow  — > 

<par> 

<switch> 

<img  src="http://www. asu.edu/lib/webcom/yuwu/VisualTour/slideshow.rp" 
region=nfuir  fill^'freeze"  system-bitrate=M450007> 

</switch> 

</par> 

</seq> 

</par> 


SMIL  offers  two  ways  to  play  media  elements:  sequentially  (one  file  right  after  another  in  a single 
region),  where  a few  files:  text,  audio,  or  video  files  can  play  at  the  same  time.  Each  line  between  the 
<par></par>  tags  is  a media  file  which  will  play  to  a specified  region.  The  <par>  tags  mean  that  media 
files  will  play  simultaneously  (in  parallel).  The  <seq>  </seq>tags  mean  that  they  will  play  sequentially. 
Fill="ffeeze"  means  that  the  final  frame  will  stay  visible  when  that  media  file  is  done. 

When  the  presentation  starts,  the  text  file  toc.rt  displays.  The  audio  clip  smone2.rmi  also  begins. 
Then  the  text  file  title. rt  clip  specified  in  the  next  line  starts  at  35’  into  the  presentation.  Then  the  slide 
show  begins.  While  the  audio  clip  is  playing,  the  text  and  slide  show  are  displayed.  By  adding  a time  line  to 
a presentation,  you  can  control  when  content  is  displayed  and  how  transitions  between  various  content 
types  are  handled.  SMIL  provides  sophisticated  control  the  protocol  offers. 

One  feature  of  SMIL  is  that  by  setting  choices,  you  can  even  serve  up  different  versions  of  the 
production  based  on  the  available  bandwidth.  In  other  words,  you  can  use  the  <switch>  tag  to  get  the  player 
to  use  media  files  appropriate  to  the  bandwidth. 

<switch> 

<img  src=Mhttp://www.asu.edu/lib/webcom/yuwu/VisualTour/slideshow.rp" 

region- 'fuir  fi  11=" freeze”  system-bitrate=n450007> 

</switch> 

The  presentation's  bandwidth  needs  are  optimized  based  on  the  speed  of  your  computer’s 
connection  to  Internet.  The  media  files  used  in  the  presentation  will  depend  on  whether  you  have  a 28.8 
modem  or  a high-speed  connection  such  as  a T1  line.  This  feature  makes  it  possible  that  users  enjoy  the 
presentation  regardless  of  their  connection  speed.  You  can  include  low-  and  high-bandwidth  choices.  The 
<switch>  tag  tells  the  player  it  should  make  a choice.  The  bandwidth  choices  are  made  in  the  order  listed, 
the  highest-bandwidth  choice  should  come  first. 

One  of  the  greatest  advantages  of  SMIL  is  synchronization.  SMIL  offers  quite  a few  features  for 
handling  the  timing  of  media  playback.  One  can  set  specifically  a duration  for  a media  clip.  For  example, 
you  can  assume  that  the  file  will  play  for  3 minutes.  You  can  time  events  according  to  the  time  line  for  the 
entire  presentation.  You  can  let  a media  file  begin  1 0 seconds  into  the  presentation  and  end  1 5 seconds  into 
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the  presentation.  You  can  decide  what  should  take  place  at  the  end  of  the  presentation.  For  instance,  you 
can  remove  all  graphics  or  video  clips  from  the  display  or  you  can  freeze  still  images  on  the  last  frame. 

You  can  use  special  effects  like  fadein,  crossfade,  viewchange,  and  wipe  target  in  your  slideshow 
presentation.  For  each  special  effect,  attributes  such  as  start  specify  the  time  to  begin,  duration  how  long 
the  picture  lasts,  and  target  the  source  of  the  picture.  For  example:  <crossfade  start="00:45" 
duration- '00 : 08 " target- ' 5 "/>. 

You'll  likely  be  using  a lot  of  file  formats  when  you  create  your  multimedia  production.  Keep  in 
mind,  you  should  transfer  all  standard  animation,  video,  and  audio  file  formats  to  streaming  formats,  such 
as  RealFlash,  RealVideo,  and  RealAudio. 

When  you  complete  presentation,  you  should  move  the  multimedia  files  and  the  SMIL  file  to  the 
web  server.  The  server  should  be  configured  to  handle  multiple  forms  of  media  including  text,  audio,  video, 
and  animation,  and  they  should  understand  hypertext  links.  If  the  server  supports  all  the  media  formats  such 
as  ra,  rt,  rm,  rp,  smil,  etc.,  you  can  view  the  presentation  in  your  SMIL  player  such  as  RealPlayer  G2, 
GRiNS  (GRaphical  iNterface  to  SMIL),  or  HPAS  (Hypermedia  Presentation  and  Authoring  System). 


Appendix:  Source  Codes  for  the  Virtual  Library  Tour: 

You  can  create  your  own  presentation  by  customizing  the  following  codes , that  is,  changing  the 
variables  and  replacing  text,  audio,  graphic  files  with  your  own. 

1.  vt. smil  code: 


<layout> 

<root-layout  height="405"  width="770M  background-color='black"/> 

<region  id- ’title"  left="5"  top- '120"  width="400"  height="200"  z- index-’ 17> 

<region  id- ’full"  left^’O"  top- ’0"  height="425"  width="450”  background  color="#6020307> 
<region  id="video"  left="450"  top="240"  height- '120”  width="315"  z-index-T7> 

<region  id="toc"  left="450"  top="0"  height="405"  width="2557> 


<audio  src=http:/Avww. asu.edu/libAvebcom/yuwu/VisualTour/smone2.rmi 
dur="400s7> 

<text  src- 'http://www.asu.edu/lib/webcom/yuwu/VisualTour/toc.rt"  region=toc’7> 

<text  src="http.V/www.asu.edu/lib/webcom/yuwu/VisualTour/song.rt"  region="video7> 
<seq> 

<!—  This  part  displays  the  title  screen  and  the  caption  with  an  audio  soundtrack--> 

<text  src="http:// www.asu.edu/lib/webcom/yuwu/VisualTour/title.rt" 
type-’text/html"  region- ’title"  dur="35s'7> 

<!--  This  section  displays  a slide  show  — > 


<smil> 

<head> 


</layout> 


</head> 

<body> 

<par> 


<par> 


<img  src=http://www. asu.edu/lib/webcom/yuwu/VisualTour/map.rp 
region- ’full"  fill="ffeeze’7> 
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</par> 

<!—  This  section  contains  the  annotated  slideshow  — > 


O 

ERIC 


<par> 

<switch> 

<img  src="http://www.asu.edu/lib/webcom/yuwu/VisualTour/slideshow.rp" 
region="fuH"  fill- ’freeze”  system-bitrate="450007> 

</switch> 


</smil> 

2.  toc.rt  code  (for  the  table  of  content): 

<window  type=generic  duration="6:30.0”  scrollrate=0  height=250  width=375  bgcolor=M#000000” 
link="#DDBBBB"  loop=true> 

<time  begin  ="00:02H/><font  size=”5”  color="white"  face=”timesM><pos  y="107> 

<br> 

<br> 

<br> 

<b>Visual  Library  Tour</b> 

<br> 

<time  begin="00:057> 

<font  size- '3"  color="white"  face=”timesM> 

<a  href="http://www.asu.edu/lib/hayden/">Hayden  Library</a></font> 

<br> 

<time  begin="00:077> 

<font  size-1 ’3"  color="white"  face- 'times"> 

<a  href=Mhttp://www.asu.edu/lib/music''>Music  Library</a></font> 

<br> 

<time  begin="00:9"/> 

<font  size="3"  color="white"  face="times"> 

<a  href="http://www.asu.edu/lib/noble/">Noble  Library</ax/font> 

<br/> 

<time  begin- '00:1  17> 

<font  size- ’3”  color=nwhiteM  face- ’times"> 

<a  href=,,http://www.asu.edu/caed/AEDlibrary/,,>Architechture  Library</a></font> 

<br/> 

<time  begin- '00: 1 37> 

<font  size- ’3”  color="white"  face- 'times"> 

<a  href=,'http://www.lawlib.asu.edu/M>Law  Library</ax/font> 

<br/> 


</par> 


</seq> 

</par> 


</body> 
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<time  begin="00: 1 5 "/> 


<font  size- '3"  color=”white”  face=,,times"> 

<a  href="http://w^w\asuxdu/iib/"xi><b>ASU  L 


ibranes</bx/iX/ a > </ font> 


</window> 


3.  title. rt  code  (for  the  title): 


<window  type-’ gen  eric"  bgcolor="#602030"  WIDTH="400"  HEIGHT="200’’> 

<center><font  size=”6"  color="white"> 

<time  begin="00:20”><center><i><b>The  Arizona  State  University  University  Libraries</b></i></font> 
<font  size- '4”  color="white"> 

<br/> 

<br/> 

<time  begin- '00:22">A  Visual  Library  Tour 
<time  begin="00:26,l><clear/> 

<time  begin=,'00:28l,><pos  y=,l50,,>Welcome  and  enjoy  a visual  experience!</center></font> 

<time  begin- ,00:30”><font  size- ’3"  color="white"xi><bxp>~  LIST,  ASU<b/x/i></fontx/center> 


</window> 

4.  map.rp  code  (for  the  slideshow): 

<imfl> 

<head  duration- '01 :05" 
width- ’450" 
height=n425M 
bitrate="  13500" 
preroll=l,00:20” 
aspect=”true” 

timeformat- ’dd:hh:mrn:ss.xyz7> 

<image  handle- '1 11  name="haydenl.jpg7> 
<image  handle- ’2"  name- ’hayden4jpg"/> 
<image  handle- '3”  name-'noble-r.gif '/> 

<image  handle="4n  name=l,musicentrance  jpg"/> 
<image  handle=”5"  name="architecturejpg”/> 
<image  handle- ’6"  name="law.jpg7> 


<fadein  start=M00:01"  duration=”00:02”  target=”l”/> 

<crossfade  start=”00:06"  duration=,,00:15”  target- ,2,,/> 

<viewchange  target=”2”  start- ’00:25"  duration- ,00:05M  srcx=”290"  srcy-’139M  srcw=”l  18” 
srch-T  157> 

<crossfade  start="00:30”  duration=”00:02”  target="3'7> 

<wipe  target="4”  start=n00:35”  duration- ’00:08”  type="normar  direction- 'down "/> 
<crossfade  start=”00:45"  duration- ’00:08”  target="5'7> 

<crossfade  start="00:55"  duration=”00: 10"  target="6’7> 

</imfl> 

</window> 
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5.  slideshow. rp  code  (codes  for  the  slideshow): 


<imfi> 

<head 

duration="04: 15*' 
bitrate="20000" 
width="450" 
height="425" 
aspect- 'true" 

time  format- 'dd:  hh : mm : ss.xyz" 
preroll~"80"  /> 

<image  handle-’ 1"  name="haydenjpg"/> 

<image  handle="2"  name="hayden2  jpgM/> 

<image  handle="3"  name- ’hayden3  .jpg’7> 

<image  handle- ’4"  name- ’hayden4jpg7> 

<image  handle=M5"  name- 'hayden5help.jpg’7> 

<image  handle- '6M  name- 'hayden6circ  .jpg"/> 

<image  handle-7’’  name="hayden7adajpgM/> 

<image  handle="8"  name="hayden8study.jpg,7> 

<image  handle=M9M  name-'hayden9ill jpg"/> 

<image  handle-’ 10”  name- ’haydenl0workstation.jpg'7> 
<image  handle=Ml  1"  name=Mhaydenl  lref.jpg7> 

<image  handle-1 12"  name- 'haydenl  2mars.jpg7^> 
<image  handle="13"  name-1 'haydenl31evell.jpg7> 
<image  handle-1 14"  name=Mhaydenl4curr.jpg'7> 

<image  handle=”15”  name="haydenl  5spcoll.jpg7> 
<image  handle="16"  name- 'haydenl6asia.jpg’7> 

<image  handle="17"  name- 'haydenl7labl.jpg'7> 

<image  handle="18"  name="haydenl8gov.jpg7> 

<image  handle- '19"  name- 'haydenl9azm.jpg'7> 

<image  handle="20"  name="hayden20ahf .jpg”/> 

<image  handle- '21"  name="hayden21  period  jpg'7> 
<image  handle=''22"  name="hayden22currperiod .jpg7> 
<image  handle=”23"  name=,'hayden23micro.jpg,7> 
<image  handIe='T'  name="hayden.jpg"/> 


<fill  start="0"  duration="00:01"  color="#6020307> 
<fadein  start="00:02"  duration- '0:03”  target="17> 
<crossfade  start="00:10''  duration="00:03"  target="27> 
<crossfade  start=”00:20"  duration=''00:03"  target="37> 
<crossfade  start="00:30"  duration="00:03"  target="47> 
<crossfade  start="00:40"  duration="00:05"  target="5"/> 
<crossfade  start="00:50"  duration- '00:05"  target- ’6’7> 
<crossfade  start="01 :00"  duration- ‘00:05"  target="7'7> 
<crossfade  start="01 :10"  duration-'00:05"  target-’8'7> 
<crossfade  start="01 :20"  duration=’’00:05"  target="9’7> 
<crossfade  start="01 :30"  duration="00:05"  target="107> 
<crossfade  start="01 :40"  duration="00:05"  target-’ 1 1 ’’/> 
<crossfade  start="01 :50"  duration=’'00:05n  target-'  127> 
<crossfade  start="02:10,,  duration="00:05"  target-’ 1 1 3’7> 
<crossfade  start="02:20"  duration="00:05’1  target- '147> 
<crossfade  start=M02:30"  duration=,'00:05,‘  target="157> 
<crossfade  start="02:40"  duration="00:05"  target="167> 
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<crossfade  start="02:50"  duration 
<crossfade  start="03:00"  duration 
<crossfade  start="03:10"  duration5 
<crossfade  start="03:20"  duration1 
<crossfade  start="03:30"  duration 
<crossfade  start="03:40"  duration1 
<crossfade  start="03:50"  duration 
<crossfade  start=,,04:00,,  duration1 


sM00:05M  target- '17"/> 
="00:05"  target-1 1 87> 
="00:05"  target- 1 1 97> 
="00:05"  target="20"/> 
="00:05"  target="21'7> 
="00:05"  target="22"/> 
="00:05"  target="23"/> 
="00:05"  target="17> 


<fadeout  start- '04: 10"  duration- '00:5"  target-' l"/> 


</imfl> 

6.  song.rt  code  (codes  for  the  caption): 

<window  type-'teleprompter"  bgcolor="black"  duration="266"xfont  color="white"> 

<br/><time  begin="36"/>Welcome  to  our  library. 

<br/><time  begin="37"/>The  Hayden  Library  is  located  on  Cady  Mall. 

<br/><time  begin=M40"/>The  Library  houses  materials  in  the  humanities  and  social  sciences. 
<br/xtime  begin="43"/>They  include  business  and  education. 

<br/xtime  begin="46"/>It  has  a number  of  specialized  collections  including  Government  Documents. 
<br/><time  begin="49"/>  1 ********************************************** 

<br/><time  begin="52"/>2  ********************************************** 

<br/xtime  begin="263 "/>********************************************** 

<br/><time  begin- '264"/>Welcome  to  our  library. 

</font> 

</window> 
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InterM ath14— Professional  and  Cognitive  Development  through 
Problem  Solving  with  Technology 


Evan  Glazer 
University  of  Georgia 


InterMath  project  group 
http://www.intermath-uga.gatech.edu/ 


How  can  teachers  teach  a mathematics  that  they  never  have  learned, 

in  ways  that  they  never  experienced? 


Cohen  and  Ball,  1990 


Abstract 


o 

ERIC 


InterMath  is  a statewide  Internet-based  (htty://www . intermat h-uga. gatech.edu/) project  with  the 
goal  of  designing  and  implementing  a series  of  workshops  and  ongoing  support  programs  that  feature 
contemporary  applications  of  technology  and  mathematics  pedagogy  in  themiddf  e-grades . Technology  is 
used  to  deliver  the  curriculum  through  web-based  materials  and  to  explore  the  mathematics  using  cognitive 
tools  such  as  dynamic  geometry’  software,  spreadsheets,  and  graphing  calculators.  Objectives  of  InterMath 
include 

• strengthening  the  middle  school  teacher's  knowledge  and  understanding  of  mathematics, 

• providing  a support  structure  (on-line  & in-school)  to  aid  teachers  in  implementing  and 
integrating  technology  tools  for  doing  mathematics,  and 

• providing  a structured  inservice  curriculum  that  follows  Georgia  fs  Quality  Core  Curriculum 
objectives  as  well  as  reform  efforts  expressed  in  publications  by  the  National  Council  of 
Teachers  of  Mathematics. 

InterMath  is  a collaborative  effort  among  the  University  of  Georgia,,  Georgia  Institute  of 
Technology >,  and  nine  regional  technology  centers  in  the  state  of  Georgia.  InterMath,  a five-year  effort  to 
design  and  implement  a series  of  field-based  workshops  and  ongoing  support  programs  to  assist  both 
teachers  and  administrators  in  effecting  mathematics  reform,  is  funded  through  the  National  Science 
Foundation. 

Rationale 

A Vision  for  School  Mathematics 

The  pedagogical  shifts  embodied  in  a series  of  documents  published  by  the  National  Council  of 
Teachers  of  Mathematics  (NCTM)  emphasize  vastly  different  approaches  to  mathematics  teaching  and 
learning  than  are  typical  in  today's  classrooms  (NCTM,  1989,  1991,  1995,  2000).  Rather  than  static 
knowledge  and  skills  detached  from  both  other  domains  and  everyday  events,  mathematics  is  viewed  as 
problem  solving,  reasoning,  and  communicating  so  that  students  are  empowered  to  confidently  "explore, 
conjecture,  and  reason  logically  [about  the  world  around  them]"  (NCTM,  1989,  p.5).  This  change  in 
learning  philosophy  reflects  a need  for  mathematics  that  is  based  in  an  information-rich  and  technology- 
based  society.  Learning  goals  should  incorporate  values  that  reflect  mathematics  for  life,  mathematics  as  a 
part  of  cultural  heritage,  mathematics  for  the  workplace,  and  mathematics  for  the  scientific  and  technical 
community  (NCTM,  2000). 


14  The  InterMath  project  has  been  funded  by  the  National  Science  Foundation  [Grant  #987661 1].  The 
views  and  opinions  of  the  authors  do  not  necessarily  represent  those  of  the  National  Science  Foundation. 
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NCTM  (2000)  suggests  that  the  direction  of  mathematics  education  should  involve  six  core 
principles:  equity,  curriculum,  teaching,  learning,  assessment,  and  technology.  The  equity  principle 
stresses  the  need  for  reasonable  expectations,  opportunities,  resources,  and  support  for  all  students  in 
learning  mathematics.  Students  should  have  access  to  different  forms  of  technology  that  will  help  them 
generate  ideas  and  support  their  thinking.  The  curriculum  principle  focuses  on  the  need  to  develop  a clear, 
coherent  plan  to  promote  important  mathematics.  Concepts  in  the  curriculum  should  relate  to  other 
mathematical  ideas  and  be  used  to  promote  mathematical  thinking  and  reasoning.  The  use  of  technology 
encourages  these  mathematical  connections  by  allowing  students  to  understand,  visualize,  and  conjecture 
about  new  or  unfamiliar  concepts. 

In  reform-based  mathematics  classrooms,  teachers  are  not  merely  keepers  and  transmitters  of 
mathematical  knowledge;  they  facilitate  student  engagement  by  posing  relevant  problems  that  encourage 
deep  mathematical  thinking  involving  analysis,  problem  finding  and  problem  solving,  that  result  in  a rich 
conceptual  understanding.  Thus,  the  teaching  principle  emphasizes  that  teachers  need  to  be  well-versed  in 
mathematics  and  pedagogy,  including  how  students  learn  mathematics  and  the  most  effective  learning 
environments,  in  order  to  fulfill  this  role.  Similarly,  the  learning  principle  emphasizes  the  need  for 
understanding  mathematical  concepts.  According  to  the  Curriculum  and  Evaluation  Standards  for  School 
Mathematics  (NCTM,  1989),  conceptual  understanding  ’’enables  children  to  acquire  clear  and  stable 
concepts  by  constructing  meanings  in  the  context  of  physical  situations  and  allows  mathematical 
abstractions  to  emerge  from  empirical  experience"  (p.  17). 

The  assessment  principle  identifies  assessment  as  a tool  for  enhancing  learning  and  informing 
instructional  decisions.  Assessment  should  support  continual  and  reflective  learning  based  on  values, 
multiple  sources  of  information,  and  feedback,  so  that  learners  take  responsibility  for  their  ideas. 
Technology  use  not  only  influences  how  and  what  mathematics  is  taught,  but  it  also  gives  students  an 
opportunity  to  construct  and  express  their  mathematical  ideas  through  their  own  creations  and 
interpretations.  However  by  itself,  "technology  is  not  a panacea"  because  any  teaching  tool  can  be  used 
poorly  (NCTM,  2000,  p.  25). 

Teachers  should  be  provided  extended  opportunities  to  experience  and  do  mathematics  in  an 
environment  supported  by  diverse  technologies  (Dreyfus  & Eisenberg,  1996).  The  development  of 
mathematical  understanding  occurs  when  technology  is  used  as  a cognitive  tool  that  supports  thinking, 
reasoning,  and  problem  solving  (Jonassen  and  Reeves,  1996).  The  use  of  cognitive  tools  such  as  dynamic 
geometry,  graphing  calculators,  spreadsheets,  and  symbolic  processors,  can  provide  opportunities  and 
experiences  for  exploration,  developing  understanding,  interpreting  and  communicating  about  mathematics 
(see  Bransford,  et  al,  1 996;  Schoenfeld,  1 982,  1 989,  1 992;  Silver,  1 987).  Our  approach  in  the  InterMath 
project  focuses  on  developing  mathematical  power— understanding,  using,  and  appreciating  mathematics. 


Barriers  to  and  Proponents  of  Reform 

Reform,  however,  does  not  occur  simply  because  new  standards  or  approaches  emerge.  Several 
barriers  have  hampered  reform  efforts.  One  barrier  appears  to  be  linked  to  resilient  and  pervasive  beliefs 
among  preservice  and  inservice  teachers  as  to  what  constitutes  mathematics  (Ball,  1988;  Dossey,  1992; 
Thompson,  1984;  1992).  Even  and  Lappan  (1994)  identified  several  widely  held  teacher  beliefs:  (1) 
computational  proficiency  is  the  major  mathematics  curriculum  goal;  (2)  mathematical  knowledge  is  rule 
bound  and  unconnected;  (3)  teaching  is  telling  and  learning  is  memorizing  (p.  129).  Howson,  Keitel,  and 
Kilpatrick  (1981)  noted  that  many  curriculum  projects  fail  because  teachers  tend  to  proceduralize  methods 
in  ways  that  are  often  inconsistent  with  the  curriculum’s  underlying  epistemological  and  pedagogical 
assumptions.  This  has  been  particularly  evident  in  the  use  of  widely  available  drill-and-practice  programs 
that  could  be  used  to  support  emerging  pedagogies,  but  rarely  are.  Cohen  (1990),  for  example,  documents 
the  activities  of  a well-intentioned  teacher  who,  based  on  lectures  about  reform  mathematics,  believed  her 
methods  were  consistent  with  the  current  reform  movement.  However,  she  never  actually  experienced 
"doing  mathematics"  or  learning  mathematics  in  these  new  ways  herself.  While  her  intent  and  motives  were 
admirable,  the  lack  of  experience  in  participating  as  a learner  inherently  limited  her  understanding  and 
insight  in  implementing  the  approaches.  In  order  to  promote  conceptual  change,  teachers  must  themselves 
experience  mathematics  as  we  want  our  students  to:  as  conjecturing,  reasoning,  communicating,  and 
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problem  solving.  Such  experiences  should  prompt  teachers  to  examine  their  fundamental  beliefs  about  such 
questions  as,  "What  is  mathematics?,"  "What  does  it  mean  to  know  mathematics?,"  "How  do  students  learn 
mathematics?,"  and  "What  is  the  role  of  the  teacher  in  the  mathematics  classroom?" 

Research  in  mathematics  education  suggests  that  a teacher’s  conception  of  mathematics  has  a 
strong  impact  on  how  mathematics  is  approached  in  the  classroom  (Cooney,  1 985;  Thompson,  1 984;  1 992). 
Furthermore,  the  nature  of  the  classroom  environment  in  which  mathematics  is  done  strongly  affects  how 
students  view  the  subject  and  how  it  should  be  taught  and  learned.  A common  theme  found  throughout  the 
reform  documents  is  "What  students  learn  is  fundamentally  tied  to  how  they  learn  it"  (NCTM,  1989,  p.  5; 
NCTM,  1991,  p.  21).  Thus,  if  we  want  our  students  to  view  mathematics  not  as  a static  body  of  rules  and 
procedures,  but  as  a meaningful  and  dynamic,  yet  connected  body  of  knowledge,  we  must  make  an  impact 
on  their  teachers'  views  of  mathematics. 

In  short,  if  we  want  our  teachers  to  meaningfully  teach  mathematics,  they  must  experience 
meaningful  mathematics.  In  the  words  of  Cohen  and  Ball  (1990),  "How  can  teachers  teach  a mathematics 
that  they  never  have  learned,  in  ways  that  they  never  experienced?"  We  cannot  expect  teachers  to  teach  in  a 
manner  consistent  with  reform  advocates  simply  because  they  have  been  told  what  to  do  or  how  to  do  it.  To 
help  our  teachers  meaningfully  teach  and  model  mathematical  thinking,  they  must  experience  relevant 
mathematics  as  learners,  benefiting  from  both  the  discovery  processes  as  well  as  guidance  from  and 
modeling  of  capable  peers.  To  break  the  cycle  of  stagnant  curriculum  and  pedagogy,  better  teacher  models 
are  needed  at  all  levels,  K-12  through  university. 


Project  Overview 
Description  and  Goals 

InterMath  (littp://www.intennath-uga.^atech.edu/)  is  a statewide  Internet-based  project  with  the 
goal  of  designing  and  implementing  a series  of  workshops  and  ongoing  support  programs  that  feature 
contemporary  applications  of  technology  and  mathematics  pedagogy  in  the  middle-grades. 

InterMath  has  two  primary  teacher  components: 

• workshops  comprised  of  in-class  portions  and  a "follow-along"  component  in  which  participants 

create  curriculum  for  use  in  their  own  classrooms. 

• an  ongoing  system  to  support  teachers  beyond  the  initial  laboratory/workshop. 

Intensive  support  will  be  provided  throughout  the  workshops  under  the  close  tutelage  of  InterMath 
facilitators  distributed  throughout  the  state.  The  site-based  component  will  focus  heavily  on  scaffolding  in- 
school reform  efforts.  As  participants  near  completion  of  the  laboratory  portion,  they  will  transition  to  the 
ongoing  support  system— a peer  community  to  ensure  continuity  beyond  the  laboratory. 

The  ongoing  support  system  is  supported  by  the  Learning  and  Performance  Support  Laboratory 
(University  of  Georgia)  and  the  Center  for  Education  Integrating  Science,  Mathematics,  and  Computing 
(Georgia  Institute  of  Technology)  where  shared  resources  and  communication  tools  are  provided; 
customization  of  support  will  be  ensured  through  distributed  implementation  sites.  Three  INTECH 
(INtegrating  TECHnology  in  the  student-centered  classroom)  centers  at  the  University  of  Georgia,  Valdosta 
State  University,  and  Kennesaw  State  University  will  be  initially  certified  as  InterMath  sites.  They  will  then 
mentor  both  subsequent  InterMath  INTECH  centers  as  well  as  serve  as  regional  support  base  for 
participants.  During  the  project,  we  will  establish  a geographically  distributed  community  of  educators,  K- 
12  through  universities,  who  are  committed  to  sustaining  technology-enhanced  middle-grades  mathematics 
teaching  and  learning  reforms.  This  community  will  be  connected  and  supported  through  shared  web- 
based  resources,  e-mail,  and  listservs. 

Project  goals  and  objectives  reflect  multiple  targets  aimed  at  involving  teachers  and  administrators 
in  technology-enhanced  mathematics  reform.  They  link  the  epistemological,  pedagogical,  and  logistical 
activities  designed  to  support  QCC  and  NCTM  standards. 
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Goal  1 : Promote  innovative  practices  in  the  tool  uses  of  technology  in  middle-grades  mathematics  teaching 
and  learning. 

• To  use  technology  toois  to  model  and  demonstrate  standards-referenced  mathematics  content  and 
pedagogy  for  the  middle  school. 

• To  enable  teachers  to  experience  mathematics  using  various  technologies  so  that  they  can  explore 
real  world  applications,  engage  in  problem  solving,  and  communicate  about  their  investigations. 

• To  use  technology  to  understand  the  distinction  between  demonstration  and  proof  in  mathematics 
and  to  emphasize  the  value  of  each  in  the  understanding  of  mathematics. 

• To  use  technology  to  engage  in  mathematics  explorations,  to  form  mathematics  ideas,  and  to  solve 
mathematics  problems. 

• To  use  technology  tools  to  construct  new  and  personally  meaningful  ideas  of  mathematics. 

• To  use  general  tools  such  as  word  processing,  paint  programs,  spreadsheets  to  facilitate 
mathematics  investigations  and  communication. 

Goal  2:  Revitalize  middle-grades  mathematics  teaching  and  learning  by  modeling,  then  applying, 
innovative  technology-enhanced  approaches. 

• To  develop  effective  mathematics  demonstrations  using  appropriate  technology  tools. 

• To  engage  in  independent  investigations  of  mathematics  topics  from  the  middle  school  curriculum 
or  from  mathematics  appropriate  for  that  level. 

• To  communicate  mathematics  ideas  arising  from  technology-enhanced  investigations. 

Goal  3 : Support  reform  of  mathematics  teaching  and  learning  in  middle  grades  mathematics  classrooms. 

• To  enable  middle  grade  mathematics  teachers  to  develop  and  adapt  materials  and  goals  from 
standards  -based  curriculum  through  the  use  of  technology. 

• To  model  and  explore  collaborative  instructional  strategies. 

• To  develop  mechanisms  and  expectations  of  sharing  instructional  ideas,  materials,  and  information 
among  middle  school  mathematics  teachers. 

• To  support  comprehensive  standards-based  middle  school  mathematics  curricula  and  the 
implementation  of  Quality  Core  Curriculum  and  NCTM  goals. 

• To  utilize  technology  tools  in  the  implementation  of  alternative  assessment  strategies. 

Goal  4:  Establish  the  human  and  technological  infrastructure  needed  to  sustain  meaningful  reform  of 
middle  grade  mathematics  instruction. 

• To  develop  confidence  in  technology  use  as  teachers  explore,  practice,  reflect,  and  become  adept 
in  technology-enhanced  teaching  and  learning  of  mathematics 

• To  enable  and  encourage  middle  school  mathematics  teachers  to  collaborate  by  using  technology 
support. 

• To  support  professional  development  opportunities  for  middle  school  mathematics  teachers  and 
other  key  personnel  through  a network  of  peer  teachers. 

Professional  Development 

Workshop  Procedures 

The  workshops  are  intended  to  immerse  teachers  in  active  problem  solving  with  technology. 
Participants  will  explore  different  concepts  each  class  meeting  by  working  through  various  InterMath 
investigations  and  writing  about  one  in-depth.  Each  participant  will  build  a personal  web  page  using 
artifacts  and  productions  from  the  workshops  to  compile  an  electronic  portfolio.  Write-ups  and  projects, 
reflecting  participants’  synthesis  and  reflection  about  their  explorations,  will  be  submitted  electronically  for 
workshop  credit.  The  purpose  and  focus  of  a write-up  is  to  communicate  and  synthesize  investigations 
involving  exploration,  solving  a problem,  or  working  with  an  application.  The  key  elements  of  a write-up 
consist  of  the  learner's  synthesis,  communication,  mathematical  ideas,  interpretation,  and  utility  of  an 
investigation.  Final  projects,  focusing  on  a technology-enhanced  mathematics  investigation  of  the 
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individual  participant’s  determination,  will  be  submitted  and  discussed  at  the  end  of  the 
workshop/laboratory.  Participant  productions  will  be  placed  on  the  web  page  for  public  sharing. 

The  laboratory  leader  will  present  demonstrations  and  explanations,  clarify  problems,  and 
demonstrate  alternative  solutions  using  a projected  image  from  the  leader’s  workstation.  In  a typical 
session,  a leader  might  allocate  one-third  of  the  time  in  whole-group  mode,  and  during  the  balance  of  the 
meeting  provide  direct  support  for  participants  working  on  their  projects  or  units,  either  individually  or  in 
groups.  The  InterMath  web  site  (http://www.intermath-uga.gatech.edu/)  will  enable  participants  to  work  at 
their  home  or  school  sites. 

In  addition  to  the  45-hour  workshop,  the  55-hour  "follow-along"  course  will  promote  the  use  of 
technology  to  enhance  mathematics  teaching  in  their  home  school  and  to  extend  each  participant's 
expertise.  This  additional  component  to  the  workshop  promotes  reflective  practice  among  the  participants, 
emphasizing  realistic  applications  of  technology  in  middle  school  teaching.  Each  participant's  web  page 
contributions  will  include  conceptual  work,  projects,  activities  for  their  classroom,  and  links  to  related 
teaching-learning  resources  in  order  to  establish  a highly  connected  framework  of  resources. 

Participant  Selection  and  Credit 

Participation  will  be  open  to  all  middle  school  educators  in  Georgia,  but  teachers  from  historically 
underserved  schools  will  receive  top  priority.  Applications  will  be  solicited  from  schools  located  within  the 
service  areas  of  the  participating  sites  (First  year:  University  of  Georgia,  Kennesaw  State  University,  and 
Valdosta  State  University). 

It  is  expected  that  a team  of  selected  teachers  will  participate  in  the  workshops  and  at  least  one 
administrator  will  participate  in  a minimum  of  20  hours  of  professional  development  as  part  of  the  team.  It 
is  also  imperative  that  each  teacher  member  has  classroom  Internet  access  and  there  is  e -mail  access  for  all 
team  members.  Teams  with  a minimum  of  one  teacher  of  mathematics  from  each  grade  level  (4-8) 
represented  in  the  school  will  receive  priority  in  the  selection  process. 

Credit  could  be  in  the  form  of  graduate  hours  or  staff  development  units  depending  on  the 
institution  offering  the  workshop.  InterMath  participants  need  to  check  with  the  site  at  which  they  will  be 
participating  to  determine  what  type  of  credit  they  will  receive.  If  graduate  credit  will  be  offered, 
participants  will  likely  have  to  apply  and  be  admitted  to  the  graduate  school  at  that  particular  institution. 

Administrative  Support 

Administrative  support  and  leadership  are  key  in  both  promoting  and  sustaining  school  innovation. 
Several  authorities  advocate  models  that  tie  professional  development  to  a particular  school  and  are 
explicitly  linked  to  reform  activities  that  the  individual  school  is  undertaking  (Darling-Hammond,  1995; 
Davis  & Padilla,  1991;  Lieberman,  1995).  Therefore,  administrative  support  in  school  improvement  plans 
influences  teacher  and  student  use  of  technology.  The  target  administrators  for  InterMath  are  building  level 
personnel  with  primary  responsibility  for  instructional  leadership,  i.e.,  principals,  assistant  principals, 
instructional  specialists.  However,  central  office  administrators  with  responsibility  for  curriculum  design, 
professional  development,  student  assessment  and  the  support  of  instructional  technology  are  also 
encouraged  to  participate  in  the  InterMath  program. 

In  the  InterMath  workshops,  administrators  will  engage  in  hands-on  activities  using  the  Internet  to 
support  teachers  in  their  efforts.  This  participation  will  ensure  that  administrators  better  understand  the 
power  and  potential  of  the  learning  activities,  the  technical  needs  of  the  teacher,  and  the  classroom 
management  techniques  that  complement  technology-enhanced  learning  experiences.  Administrators  will 
be  encouraged  to  share  their  ideas,  problems,  solutions,  and  successes  for  supporting  their  teachers.  Thus, 
InterMath  workshops  can  provide  a forum  to  reduce  administrator  isolation  and  support  administrators  in 
follow-up  activities  including  instructional  leadership  and  teacher  evaluation. 
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Our  Use  of  Technology 

The  InterMath  workshop  intends  to  illustrate  how  and  when  technology  can  be  used  appropriately 
in  the  mathematics  classroom.  The  literature  describes  two  distinctly  different  approaches  in  the  use  of 
technology  in  classrooms:  using  the  computer  as  a tool  for  exploration  or  problem  solving  and  using  the 
computer  as  a tutor  that  delivers  instruction  and  provides  feedback.  Research  on  the  use  of  computers  in 
mathematics  as  a tutor  and  a tutee  are  usually  not  situated  in  problem  solving  environments.  Most  tutor- 
based  technologies  are  in  the  form  of  drill  and  practice  software,  which  tend  to  rely  on  lower  ordered  skills, 
and  are  often  negatively  related  with  student  achievement  (Jonassen  & Reeves,  1996;  Wenglinsky,  1998). 
Jonassen  and  Reeves  (1996)  argued  that  higher-order  thinking  occurs  in  environments  where  the  student  is 
learning  with,  and  not  from,  the  computer.  It  is  this  approach  that  InterMath  promotes  and  intends  to 
develop  among  its  participants. 

Many  studies  investigating  technology-enhanced  environments  include  an  emphasis  on  conceptual 
development  situations.  For  example,  when  calculators  and  computer  software  perform  calculations  and 
simplifications,  teachers  have  more  time  to  emphasize  why  something  is  happening,  instead  of  focusing  on 
algorithms  (Grassl  & Mingus,  1997;  Heid,  1988;  Maury,  1987;  Palmiter,  1991).  Moreover,  the 
imperfections  in  calculator  graphs  and  computations  also  provide  opportunities  for  conceptual 
development.  For  example,  Dion  (1990)  found  cases  where  the  graphing  calculators'  resolution  caused 
certain  functions  to  appear  differently  than  they  are  supposed  to.  In  addition,  Goldenberg  ( 1 998)  found  that 
the  graphing  calculator  window  can  provoke  critical  inquiry  because  different  functions  can  appear  to  look 
the  same  if  they  are  on  different  domain  and  range  windows.  Finally,  Burrill  (1992)  noticed  that  the 
calculator  has  difficulty  simplifying  computations  with  extremely  large  and  small  numbers,  consequently 
producing  an  incorrect  answer.  Used  appropriately,  these  situations  expose  misconceptions  and  help 
students  develop  a richer  understanding  of  the  mathematics  being  studied. 

Technology  Applications  and  Facilities 

The  technologies  used  in  the  InterMath  program  range  from  low-end,  hand-held  calculators 
through  high-end  multimedia  workstations.  Computer  software  applications  including  spreadsheets, 
graphing  tools,  dynamic  geometry,  web  editing,  and  Internet  will  be  used  regularly  throughout  the 
workshops.  Technology  will  be  available  and  supported  both  at  INTECH  sites  and  in  the  participants' 
schools.  All  INTECH  hbs  have  high-speed  Internet  access  to  support  individual  workstation,  local 
network,  and  web-based  mathematics  activities  and  applications.  The  laboratory  also  affords  ready  access 
to  non-computer  technologies,  including  graphing  calculators  and  manipulative  materials. 


Cognitive  Development 
Rationale  of  Workshop  Activities 

In  designing  the  workshops,  we  have  kept  in  mind  the  work  of  Malone  and  Lepper  (1987) 
concerning  the  design  of  instructional  environments  that  are  intrinsically  motivating.  They  have  identified 
four  sources  of  intrinsic  motivation  in  learning  activities:  (1)  gives  an  appropriate  level  of  challenge,  (2) 
appeals  to  the  sense  of  curiosity,  (3)  provides  the  learner  with  a sense  of  control,  and  (4)  encourages  the 
learner  to  be  involved  in  a world  of  fantasy  in  which  learners  can  experience  vicariously  rewards  and 
satisfactions  that  might  not  be  available  to  them  otherwise.  While  a workshop  leader  may  not  be  able  to 
incorporate  all  of  these  sources  of  intrinsic  motivation  into  every  learning  activity,  incorporating  at  least 
one  appears  to  increase  the  likelihood  that  the  activity  will  be  intrinsically  motivating. 

Pertaining  to  the  first  source  of  intrinsic  motivation,  we  have  included  a variety  of  problems  on  a 
continuum  of  difficulty  levels.  By  posing  challenging  problems  within  a familiar  context,  teachers  will 
develop  confidence  in  problem  solving  and  thus  will  more  likely  engage  in  the  activities.  The  context  of  the 
problems  enables  teachers  to  safely  sample  and  reflect  on  their  own  approaches  to  problem  solving.  The 
second  source  of  intrinsic  motivation  is  appealing  to  the  sense  of  curiosity.  Activities  can  stimulate 
curiosity  by  introducing  ideas  that  are  surprising  or  discrepant  from  the  learner’s  existing  beliefs  and  ideas. 
While  the  mathematical  problems  posed  in  the  laboratories  will  center  on  middle-school  curriculum,  they 
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are  more  open-ended  and  generative  than  is  typically  seen  in  a traditional  middle-school  curriculum. 
Problems  can  be  used  as  a springboard  for  ideas  and  investigations  that  participants  find  personally 
intriguing.  Furthermore,  teachers  wiii  be  abie  to  choose  among  several  activities  in  which  to  actually 
engage.  They  can  choose  activities  that  are  most  applicable  to  their  classroom  needs  and  relevant  to  their 
mathematical  understanding.  Since  participants  can  choose  activities  based  on  their  preferences,  the  third 
source  of  intrinsic  motivation  (providing  the  learner  with  a sense  of  control)  will  be  reflected  throughout 
the  laboratory. 

The  fourth  source  of  intrinsic  motivation  is  encouraging  engagement  through  fantasy.  As  an 
example  of  a task  using  fantasy,  consider  the  following  problem  requiring  the  use  of  the  Pythagorean 
theorem: 

The  learner  needs  to  calculate  the  distance  from  point  a to  point  b in  order 
to  inform  Captain  James  T.  Kirk  about  how  to  set  the  transformer  beam  on  the 
Federation  Starship  Enterprise  so  they  can  pick  up  the  necessary  dilithium  crystals 
directly  below  on  the  planet's  surface.  Kirk  only  knows  the  distances  of  the  ship  and 
the  crystals  from  a third  point  where  his  scouting  party  has  stopped  (Lepper  & 
Hodell,  1980). 

Fantasies  are  more  intrinsically  motivating  when  they  employ  characters  and  situations  with  which 
the  learner  can  identify.  Faced  with  either  this  fantasy-like  problem  or  a series  of  abstract  problems  in 
which  learners  are  asked  to  find  the  length  of  one  side  of  a triangle,  one  can  imagine  which  type  of  problem 
learners  would  prefer. 

The  philosophy  permeating  InterMath  is  that  teachers  must  relearn  mathematics  in  a more 
open-ended,  generative  manner  so  they  may  come  to  understand  what  reform  documents  intend  by 
"meaningful  learning."  Furthermore,  by  encouraging  teachers  to  create  and  modify  their  own  curriculum 
units,  InterMath  attempts  to  avoid  what  Howson,  et  al.  (1981)  warn  may  be  a cause  for  failed  reform  - 
teachers  failing  to  assume  ownership  of  reform. 

Workshop  Content 

The  mathematics  content  and  concepts  of  InterMath  reflect  curriculum  that  would  enhance  a 
teacher's  understanding  of  middle-grades  mathematics.  The  laboratory  centers  on  the  middle-school 
mathematics  curriculum  per  Georgia’s  Quality  Core  Curriculum  (QCC)  and  the  NCTM  Standards  (1989, 
2000).  The  InterMath  curriculum  is  meant  to  engage  teachers  and  is  intended  to  deepen  teachers' 
understanding  of  mathematical  concepts  related  to  the  middle  school  curriculum.  Thus,  the  investigations 
would  likely  need  to  be  modified  for  use  with  middle  school  students. 

There  are  14  units  that  can  be  used  for  InterMath  workshops.  Thirteen  units  are  called  Fraction 
and  Decimals,  Integers;  Ratios,  Proportions  & Percents;  Quadrilaterals,  Triangles,  Polygons,  Probability, 
Statistics,  Solids,  Circles,  Graphs,  Patterns,  Functions  & Equations.  The  fourteenth  unit  is  comprised  of 
over  200  problems  adapted  from  Teaching  Mathematics  in  the  Middle  School  (NCTM).  The  following 
criteria  have  been  used  to  highlight  recommended  investigations  for  teacher  exploration: 


• Multiple  cases  can  be  investigated  using  technology. 

• Pre-Algebra  students  can  rely  on  technology  to  investigate  the  situation. 

• The  investigation  promotes  generalizability  or  can  be  used  as  a springboard  for  further 
exploration. 

• Multiple  methods  can  be  used  to  explore  the  situation. 

• Multiple  solutions  are  possible. 

• The  investigation,  based  on  middle  school  mathematics,  is  easy  to  start  exploring. 

• The  investigation  can  be  modified  for  use  in  a middle  school  classroom. 

The  following  investigation  exemplifies  these  principles: 
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Choose  two  numbers.  Add  them  together  and  form  a Fibonacci-like  sequence,  starting  with  your 
first  two  numbers,  and  ending  with  a total  of  ten  numbers.  For  example,  if  your  first  two  numbers 
are  3 and  5,  then  your  third  number  is  8 (3+5),  your  fourth  number  is  13  (8+5),  your  fifth  number 
is  21  (8+13),  and  so  on.  Determine  a relationship  using  the  seventh  term  and  sum  of  the  terms  of 
your  sequence.  Is  this  true  for  every  sequence  of  this  nature?  Explain. 

• Multiple  cases  can  be  investigated  using  technology. 

The  use  of  formulas  in  a spreadsheet  allows  teachers  to  change  the  initial  two  numbers  and 
instantly  view  calculations  of  the  remaining  8 terms  and  the  sum  of  the  sequence.  As  multiple 
cases  are  tested,  a formula  relating  the  seventh  term  and  sum  of  the  sequence  can  be  hypothesized, 
tested,  and  modified. 

• Pre-Algebra  students  can  rely  on  technology  to  investigate  the  situation. 

While  students  can  conceivably  make  a reasonable  conjecture  about  this  investigation  using  a few 
cases,  they  will  need  an  algebraic  proof  to  verify  that  their  conjecture  is  true.  The  use  of 
technology  in  this  case  amplifies  the  confidence  in  their  conjecture  because  multiple  cases  can  be 
tested. 

• The  investigation  promotes  generalizability  or  can  be  used  as  a springboard  for  further  exploration. 

Following  the  experimentation  process  with  technology,  teachers  are  encouraged  to  question  why 
a particular  pattern  develops  and  then  investigate  a proof.  Furthermore,  after  answering  the  initial 
question,  teachers  may  develop  further  questions,  such  as: 

1 . Will  this  hypothesis  be  true  for  negative  numbers? 

2.  Will  this  hypothesis  be  true  for  decimals  and  fractions? 

3.  Are  there  relationships  between  the  sum  and  other  terms  in  the  sequence? 

• Multiple  methods  can  be  used  to  explore  the  situation. 

A spreadsheet  can  be  used  in  a variety  of  different  ways  to  investigate  this  situation.  For  example, 
students  can  construct  one  table  that  continually  changes,  or  a grid  with  multiple  tables.  In 
addition  to  spreadsheets,  symbolic  manipulation  can  be  used  to  investigate  this  problem. 

• Multiple  solutions  are  possible. 

Many  people  will  propose  that  1 1 times  the  7,h  term  will  equal  the  sum  of  the  sequence.  In 
addition,  multiple  linear  combinations  are  also  acceptable,  such  as  six  times  the  7th  term  plus  2 
times  the  9,h  term  minus  the  4th  term  will  equal  the  sum  of  the  sequence. 

• The  investigation,  based  on  middle  school  mathematics,  is  easy  to  start  exploring. 

Only  basic  arithmetic  operations  are  used  in  this  investigation.  Most  people  begin  using  positive 
integers  in  their  exploration,  and  then  later  broaden  their  scope  to  different  types  of  numbers  such 
as  negative  integers,  fractions,  decimals,  and  irrational  numbers. 

• The  investigation  can  be  modified  for  use  in  a middle  school  classroom. 

This  investigation  can  be  immediately  adopted  in  the  middle  school  classroom  if  the  i ntention  is  to 
teach  pattern  recognition  and  creation  of  formulas  from  data.  However,  a shorter  sequence  might 
be  used  in  the  classroom  if  the  teacher  intends  to  illustrate  adding  like  terms,  using  the  distributive 
property,  and  creating  linear  combinations  with  variable  expressions. 

Next  Steps 

InterMath  is  in  its  second  year  of  a five  year  project.  Over  the  past  year,  the  web-based  InterMath 
materials  have  been  developed  and  tested  with  various  teachers  in  the  state  of  Georgia.  This  year  the 
project  will  run  workshops  in  the  spring  and  summer  semesters  to  build  a community  of  teachers  that  will 
develop  technology-enhanced  materials  for  their  classrooms.  In  addition,  we  intend  to  develop  an  ongoing 
support  system  that  will  encourage  a sustained  effort  among  teachers  in  the  InterMath  program.  The  goal 
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at  the  end  of  the  five  year  project  is  to  have  a self-sustaining  system  of  resources,  tools,  and  people  with  a 
common  goal  of  enhancing  mathematics  education  using  technology  as  a catalyst  for  change. 
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